Was getting segfaults with multiple services using
`x-develop` and `watch` at the same time. Turns out
the Moby path matcher lazily initializes the regex
pattern internally the first time it's used, so it's
not goroutine-safe.
Change here is to not use a global instance for the
ephemeral path matcher, but a per-watcher instance.
Additionally, the data race detector caught a couple
other issues that were easy enough to fix:
* Use the lock that's used elsewhere for convergence
before manipulating
* Eliminate concurrent map access when triggering
rebuilds
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This approach mimics Tilt's behavior[^1]:
1. At sync time, `stat` the path on host
2. If the path does not exist -> `rm` from container
3. If the path exists -> sync to container
By handling things this way, we're always syncing based on the true
state, regardless of what's happened in the interim. For example, a
common pattern in POSIX tools is to create a file and then rename it
over an existing file. Based on timing, this could be a sync, delete,
sync (every file gets seen & processed) OR a delete, sync (by the
the time we process the event, the "temp" file is already gone, so
we just delete it from the container, where it never existed, but
that's fine since we deletes are idempotent thanks to the `-f` flag
on `rm`).
Additionally, when syncing, if the `stat` call shows it's for a
directory, we ignore it. Otherwise, duplicate, nested copies of the
entire path could get synced in. (On some OSes, an event for the
directory gets dispatched when a file inside of it is modified. In
practice, I think we might want this pushed further down in the
watching code, but since we're already `stat`ing the paths here now,
it's a good place to handle it.)
Lastly, there's some very light changes to the text when it does a
full rebuild that will list out the (merged) set of paths that
triggered it. We can continue to improve the output, but this is
really helpful for understanding why it's rebuilding.
[^1]: db7f887b06/internal/controllers/core/liveupdate/reconciler.go (L911)
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Big change here is to import the ephemeral ignore set from Tilt.
The `.git` directory is also ignored for now: this restriction
should probably be lifted and made configurable in the future,
but it's not generally important to watch and triggers a LOT of
events (e.g. Git creates `index.lock` files that will appear and
disappear rapidly as terminals/IDEs/etc interact with Git, even
for read-only operations).
The Tilt-provided ephemeral file set has been slowly devised over
time based on temporary files that can cause trouble. We can also
look at a more robust/configurable solution here in the future,
but thse provide a reasonable out-of-the-box configuration for
the moment.
There's also some small tweaks to the output to add missing
newlines in a few edge cases and such.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Nicolas De Loof
Milas Bowman
Luis Rascao
Laura Brehm
Guillaume Lours
Ben Moss
maxcleme
Laura Brehm