It is a good thing I decided to rewrite my thesis project from the ground up (almost). I'm finding that many of the assumptions that were made in order to "get it done" were thread ignorant. For example, I had assumed that system calls like chdir() were thread safe - they aren't always.
Take for example the innocent looking chdir() call. Seems safe enough. However, the 'current working directory' is a process resource not a thread resource. Changing the directory in one thread changes the working directory in all the threads. Two threads calling chdir() at the same time produces a race condition on the shared variable. Bad, bad, bad. This affects how I have to handle directory changes in the client threads and possibly means a change in the system's file structure design.
This wasn't a problem in the alpha system since there were only two possible threads: a worker thread and a server thread. As it turned out, these threads were mutually exclusive. Obviously, this wasn't the desired architecture. The final design called for a much more robust implementation, allowing any number of worker threads to run concurrently with various server, management, and service threads. If I had chosen to keep trudging along with expanding the alpha version, I would be chasing these problems down and not understanding where they came from. It is much better to face them now.
The beta version of the system is already 100 times better than the alpha code. It will be better still.
chdir is a process resource.
