Weekly progress

This week I started to refresh up my knowledge about C language and glibc apis. Wrote a simple echo tct server to start using poll/epoll.

Then I dived  directly into the fuse filesystem (llibfuse) sources. Most of the high level api in fuse.h is pretty basic, and its pretty well documented. There is also a low level fuse api which is not very well documented and the high level one is just a layer on top of the low layer one. The high level apis operate on file paths whereas low level apis directly deals with inodes. Another major difference is that high level api requires you to do the whole file system operation in a single synchronous function whereas low level api operates on a callback style api and allows for async processing.

For the fuse filesystem project work I have created a new github repo, for now I have named it fucache. Some notes about the high level fuse implementation over the low level apis are included in the notes folder in the repo.

It would be a lot more appropriate to use the low level apis for the caching filesystem as you really wanna cache the inodes rather than the file paths. The only problem however is that the high level api gives you a lot of benefits and optimizations for free which are not there in the low level api, surprisingly the code in libfuse for high level api is pretty readable and shouldn’t be hard to understand how the optimisations are implemented.

For now folks it’s basically been caching up with docs and getting some initial momentum, I am already up to speed and there should be a lot more progress next week. For now I plan to rewrite the proxyfs using the low-level api (should require a lot of bootstrap code) and take over the command line handling completely from fuse. I also plan out using different caching techniques (letting the kernel cache everything and advising it directly using mmap apis, or taking over the memory management altogether and talking directly using DIRECT_IO). The low level fuse apis actually allow the option for zero copy fs operations (using pipe fds as buffer and using splice to copy over to fd) which is pretty awesome and opens up a lot of options to implement the caching layer.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s