Do file watchers registered in the main thread get called and then enqueue a message to a worker thread for processing?
I am guessing you want to keep the code that handles file events on the watcher and on startup the same code used in two places.
Guessing you scan multiple source directories of files recursively.
Does C# have a thread safe queue object? You could create a pool of worker threads and the file watcher can enqueue events
You could have threads that scan file sources (one per source) which enqueue file names to worker threads which do the work. You could have a queue per source thread and worker thread.
The problem with the file watcher code is that I don't know what context that event runs in, so you would either have to enqueue events from the main thread context to one of the worker thread queues.
> Do file watchers registered in the main thread get called and then enqueue a message to a worker thread for processing?
Yes, Producer Consumer pattern. Currently a single thread each, but that would be scalable later. For now I try to keep things simple.
> Guessing you scan multiple source directories of files recursively. Does C# have a thread safe queue object? You could create a pool of worker threads and the file watcher can enqueue events
Yes. There are a few. I use BufferBlock<T> [1], which is pretty flexible.
> The problem with the file watcher code is that I don't know what context that event runs in, so you would either have to enqueue events from the main thread context to one of the worker thread queues.
This is the long term plan. Using events is much more flexible than "polling" the next batch of file items (even if it is in realtime). The architechture seems to work out for this but I think for now I'm pretty close to a working solution. Maybe I start going for it, develop a small UI in flutter and see, where there might be problems :-) Currently there is too much "theory" - I would like to see this in practise.
I am guessing you want to keep the code that handles file events on the watcher and on startup the same code used in two places.
Guessing you scan multiple source directories of files recursively.
Does C# have a thread safe queue object? You could create a pool of worker threads and the file watcher can enqueue events
You could have threads that scan file sources (one per source) which enqueue file names to worker threads which do the work. You could have a queue per source thread and worker thread.
The problem with the file watcher code is that I don't know what context that event runs in, so you would either have to enqueue events from the main thread context to one of the worker thread queues.