wingolog

correct or inotify: pick one

21 May 2018 2:29 PM (inotify | concurrency | linux | unix | igalia)

Let's say you decide that you'd like to see what some other processes on your system are doing to a subtree of the file system. You don't want to have to change how those processes work -- you just want to see what files those processes create and delete.

One approach would be to just scan the file-system tree periodically, enumerating its contents. But when the file system tree is large and the change rate is low, that's not an optimal thing to do.

Fortunately, Linux provides an API to allow a process to receive notifications on file-system change events, called inotify. So you open up the inotify(7) manual page, and are greeted with this:

With careful programming, an application can use inotify to efficiently monitor and cache the state of a set of filesystem objects. However, robust applications should allow for the fact that bugs in the monitoring logic or races of the kind described below may leave the cache inconsistent with the filesystem state. It is probably wise to do some consistency checking, and rebuild the cache when inconsistencies are detected.

It's not exactly reassuring is it? I mean, "you had one job" and all.

Reading down a bit farther, I thought that with some "careful programming", I could get by. After a day of trying, I am now certain that it is impossible to build a correct recursive directory monitor with inotify, and I am not even sure that "good enough" solutions exist.

pitfall the first: buffer overflow

Fundamentally, inotify races the monitoring process with all other processes on the system. Events are delivered to the monitoring process via a fixed-size buffer that can overflow, and the monitoring process provides no back-pressure on the system's rate of filesystem modifications. With inotify, you have to be ready to lose events.

This I think is probably the easiest limitation to work around. The kernel can let you know when the buffer overflows, and you can tweak the buffer size. Still, it's a first indication that perfect is not possible.

pitfall the second: now you see it, now you don't

This one is the real kicker. Say you get an event that says that a file "frenemies.txt" has been created in the directory "/contacts/". You go to open the file -- but is it still there? By the time you get around to looking for it, it could have been deleted, or renamed, or maybe even created again or replaced! This is a TOCTTOU race, built-in to the inotify API. It is literally impossible to use inotify without this class of error.

The canonical solution to this kind of issue in the kernel is to use file descriptors instead. Instead of or possibly in addition to getting a name with the file change event, you get a descriptor to a (possibly-unlinked) open file, which you would then be responsible for closing. But that's not what inotify does. Oh well!

pitfall the third: race conditions between inotify instances

When you inotify a directory, you get change notifications for just that directory. If you want to get change notifications for subdirectories, you need to open more inotify instances and poll on them all. However now you have N2 problems: as poll and the like return an unordered set of readable file descriptors, each with their own ordering, you no longer have access to a linear order in which changes occurred.

It is impossible to build a recursive directory watcher that definitively says "ok, first /contacts/frenemies.txt was created, then /contacts was renamed to /peeps, ..." because you have no ordering between the different watches. You don't know that there was ever even a time that /contacts/frenemies.txt was an accessible file name; it could have been only ever openable as /peeps/frenemies.txt.

Of course, this is the most basic ordering problem. If you are building a monitoring tool that actually wants to open files -- good luck bubster! It literally cannot be correct. (It might work well enough, of course.)

reflections

As far as I am aware, inotify came out to address the needs of desktop search tools like the belated Beagle (11/10 good pupper just trying to get his pup on). Especially in the days of spinning metal, grovelling over the whole hard-drive was a real non-starter, especially if the search database should to be up-to-date.

But after looking into inotify, I start to see why someone at Google said that desktop search was in some ways harder than web search -- I mean we all struggle to find files on our own machines, even now, 15 years after the whole dnotify/inotify thing started. Part of it is that the given the choice between supporting reliable, fool-proof file system indexes on the one hand, and overclocking the IOPS benchmarks on the other, the kernel gave us inotify. I understand it, but inotify still sucks.

I dunno about you all but whenever I've had to document such an egregious uncorrectable failure mode as any of the ones in the inotify manual, I have rewritten the software instead. In that spirit, I hope that some day we shall send inotify to the pet cemetery, to rest in peace beside Beagle.

86 responses

  1. Anon says:

    There is also fanotify https://lwn.net/Articles/339399/

    I'm not sure it solves any problem for you, but IIRC it was create to overcome problems with inotify

  2. Jos says:

    The state of watching file system events without needing priviliges is indeed lamentable. You've not even mentioned the insane amount of memory needed to keep inotify monitors on all directories: userspace is responsible for mapping the fd in the event back to the full path. So the client has to keep all directory paths in memory.

    An efficient kernel interface for this is really needed. Even with fast SSD, the overhead of doing find is still high.

    find ~ -printf '%T@\t%p\n'|wc -l

    on a home dir with 1.5M files takes 35 seconds on my laptop with fast SSD.

  3. Michael Catanzaro says:

    Michael Kerrisk discusses many of these problems in https://lwn.net/Articles/605128/.

  4. Helge Bahmann says:

    I don't think inotify (or fanotify FWIW) was intended for the purposes you describe ("see what some other processes on your system are doing to a subtree of the file system"). It was intended to allow applications to catch up with the *state* of a fileset (or tree) in an "eventually consistent (!)" fashion. As such, it is not intended to (and cannot) replace active scanning/reading of filesystem state, but to cut down on the rate/scope of when/what to scan. Basically: "no event" -> just sleep. "any event" -> rescan such that we have covered everything potentially affected by last event (worst case whole tree).

    For activity trace with temporal ordering it is more efficient to trace the actors (=processes) instead of the objects (=files) because there are far fewer. No file-based notification API can give you that.

  5. Marc says:

    Yeah I think you're confusing "inotify doesn't work" with "inotify doesn't scale" (to multiple directories, processes, etc.)

  6. ak says:

    Yeah! Wow Its amazing game which name unblocked game junction. Its superb because concept of playing and supporting are fabulous. Its speechless thank god.

  7. Thesis writing service says:

    This is an amazing knowledge about computer processors and file system. Computer is the sign of big growing of technology.

  8. Essay help UK says:

    Presently, I know I can set a cron occupation to do as such consistently, yet is there a Like the minute the document is made or changed, something lifts up on it and Taking a gander at inotify, and checking whether it's something near what I need.

  9. Coursework writers uk says:

    If you can assign space adequate to store the event(s) at the same period that you appeal notification, mistake treatment is easy and clear from a project view.

  10. Margaret Watkins says:

    Hello. Thank you very much for the article. It is very important for me.
    For my work, the most important thing is my laptop. After all on it I write various essays for the site https://writemypaper4me.co/.
    But lately I've noticed that the computer has become very hot. I do not think that the ordinary Libre office on my Linux system loads the system or Chrome.
    So thanks, I will now know what to do next time ...

  11. app promotion services says:

    The iOS platform has maintained its current status no matter rivals, which are intimidating making it extraneous with cheaper options. Even though smartphones are growing in large figures, iPhone remains a regular device since it is connected getting a classy amount of people. For this reason reason, iOS apps are seen to be lucrative if they are launched and marketed inside an effective manner. Really the only factor, that you have to keep in mind is not however the obvious method of attracting users for installing your iOS application. It is a skill, which should be learned. That’s how you're able to learn how to increase iOS application downloads while using the given guides.

  12. 70-347 Exact2pass says:

    Thank you very much for this valuable contribution and informative.

  13. cheap essay writing service uk says:

    In data security and programming, a cushion flood, or support invade, is an oddity where a program, while composing information to a cradle, overwhelms the cradle's limit and overwrites adjoining memory areas.

  14. JAVA ASSIGNMENT HELP says:

    Online assignments help services are run by experts and professionals. They are available 24/7. Students can access these websites and services anytime. They provide full access to every problem and solve the problem in dedicated time. Students can save a lot of time and effort. Taking online services is a wise choice by which students can enroll their self in other activities and utilize their time more effectively.

  15. Andrea says:

    I think you should manually approve or use a stronger CAPTCHA for comments that provide an URI/URL as the last few seem to be link farming spam... ^__^;

  16. hole io says:

    wow! These look perfect!! this is truly an amazing article

  17. Alex Dixon says:

    Great article! Very interesting to read.

  18. write my essay says:

    Losing sleep over unfinished essays but can't afford the assistance of costly essay writers? Wondering whether the online service providers will write my essay cheap in UK? Facing a deadline for finishing the essay? Well, your anxieties end here. We have with us a team of essay writing services for UK students who can solve all your writing issues so that you score high grades in your essay assignment. They will deliver the completed essay to you well before the deadline. With decades of experience in helping students, this team of professional essay typers best in UK.

  19. Assignment Help says:

    Thanks for sharing the information keep on sharing new things.

  20. Speedypaper says:

    Useful post. Thanks!

  21. Assignment Help says:

    This post is not just informative but impressive also. The post is so convincing that it created an urge to choose Assignment Help services. You can email us at Info@Myassignmenthelpau.Com or Phone Number: +61-2-8005-8227

  22. Assignment Help says:

    I loved this post and the way the author has explained the points. I am sure that any student after reading this will be compelled to search for online assignment help.
    I am a freelancing subject matter expert with Assignment Help Australia and My Assignment Help OZ. You can email us your query at

  23. Ignou MBA Project says:

    I have used the notify and I am totally satistfied with this function. We have already tested this program in our company Ignou mba project

  24. kris martin says:

    Writing a project is not an easy task specially when you have to take the approval on that because student need to do research and survey before writing the project.

  25. ignou solved assignment says:

    Get Your IGNOU Assignments for session 2018, which are solved by specialists under IGNOU guidelines are accessible on the web, IGNOU Solved Assignments of all Courses and Subjects of B.A, B.Com, B.Sc, M.A, M.com, MBA and more for the session 2018 - 2019, can be easily downloaded in PDF subsequent to submitting the request of IGNOU Solved Assignment 2018 -2019 effectively.

  26. Assignment Help says:

    Thanks for sharing such a nice piece of information to us. This is very knowledgeable for me. I am John and i am offering Assignment Help to students of Australia and all over the world.

  27. CDR Assignment Help says:

    How Can Students be Saved from Rejection with the Help of CDR Report Writers at AssignmentStudio.net?

  28. World War Rising on PC says:

    best site ever

  29. Acer says:

    i found it so amazing

  30. Wattpad : Free Books on PC says:

    it so interesting

  31. Parcheesi Classic For PC says:

    love it

  32. AptekaKamagra says:

    This is very important and it will be difficult to do anything without it.

  33. Chasing car speed drifting for PC says:

    such a nice blog

  34. Best Editing Services says:

    Thanks for sharing such awesome article. In my views, the points that you had mentioned above are more than enough and will help me allot. I have save your site and definitely visit here again!

  35. Live or Die: Survival Pro for PC says:

    love it

  36. DRAGON NEST M FOR PC says:

    i found it quite interesting

  37. Assignment Help says:

    My Assignment Services constantly aim to expand our base of assignment help and call in international experts who are ex-professors from reputed business schools, management schools, engineering universities from across the globe.

  38. Finance Assignment Help says:

    My Assignment Services do not make empty promises and have always achieved what we promised our clients by online finance assignment help.

  39. history assignment help says:

    Get expert help for your History Assignments at affordable prices. We deliver quality assignments on time every time.

  40. MYOB assignment help says:

    We offer exceptional MYOB perdisco assignment helpers at affordable prices. So, to get 100% original assignment at affordable prices via GotoAssignmentHelp.

  41. Connectify Hotspot Crack says:

    Connectify Hotspot Crack is the paramount software program which is making your virtual PC to the WiFi hotspot. It is a valuable apparatus you can without much of a stretch offer your internet connection to different devices, for example, systems, Android telephone, Mac, and iPhone. Therefore, you effectively share your wireless connection to the next PC inside a sensible separation.

  42. writing essays says:

    The blog post takes a look at the correct or inotify. The article says that ‘Linux gives an API to allow a process to receive notifications on file-system change events, called inotify. Fundamentally, inotify races the monitoring process with all other processes on the system’. If you don’t have a better idea about the inotify, you can get to it by reading the article.

  43. College Essay Help says:

    Thanks for sharing such a nice piece of information to us. This is very knowledgeable for me.

  44. Stefan says:

    Lots of wrong information.

    For a start, you confuse watch descriptors and file descriptors.

    Second, if you want to track directory renames, use IN_MOVE_FROM, IN_MOVE_TO and IN_MOVE_SELF and combine them using the event cookies.

  45. social says:

    Thanks for the great post keep up the amazing work

  46. Online Assignment Help says:

    Online Assignment Expert is a Branding Assignment provider where students get Online assignment help for all type of subjects like, economics assignment hep, finance assignment help, marketing assignment help, and more with the help of our experts. Expert’s consultation is also available for students. If they have any query related to assessment of any type you can contact with our experts anytime.

  47. 192.168.0.1 says:

    inotify came out to address the needs of desktop search tools

  48. One euro Hosting says:

    That's a great picking of the topic to continue.

  49. happy wheels game says:

    I could get by. After a day of trying

  50. Ajay Gorecha says:

    Nice Please Visit https://www.digitalhealthtrend.com/

  51. Joihn says:

    If want hacking Please visit https://www.hackingum.us

  52. Aurushi Dutta says:

    hey, I am looking for free tutorial in picsart photo editing so i get on this article thank you so much...

  53. Hackingum says:

    Best Tips And tricks

  54. Dental implant clinic in india says:

    I am happy for remarking to give you a chance to know about what an ideal ordeal my companion's young lady created experiencing your blog. She found numerous pieces, which included what it resembles to have a viable.

  55. Dissertation Writing Services says:

    Picking out all of these things, this must be notified to all of the people to maintain the data correctly and I am sure that your guidelines will be the best choice regarding this.

  56. Assignment help says:

    I am Andy, a tech savvy and academic writer by profession in StudentAssignmentHelp.com and photographer by passion.I am in a team of 7 years experienced assignment helpers who provides best academic aid for worried students.We at StudentAssignmentHelp.com are 7 years experienced assignment helpers and provide best academic aid for worried students.

  57. case study help says:

    Gotoassignmenthelp is the best site for all kind of online case study help because, for case studies writers must have abundant knowledge on the topic, which our professional case study writers definitely possess.

  58. Write My Essay, Essay Help, Essay Writing Help, Do My Essay UK says:

    Do not be afraid . A decision should never have any kind of coercion. But you know that coercion, direct or indirect, is present in many of the requests made to you. People usually say yes because they are afraid of losing their job, losing friends, losing the favoritism of a father or mother in front of their brothers ... Saying yes out of fear is gradually losing your self-confidence and your self-esteem. Knowing how to say that it is not, in some way, learning to renounce something and that resignation will always be accompanied by uncertainty, an uncertainty that you must avoid that transforms into fear.

  59. Essay Online Store says:

    I'm a freelancer in a writing company and I can say you that your writing skills are good, You can be a fascinating writer. If you want I can give you some meaningful pieces of advice.

  60. Dell Printer Support says:

    Dell Printer Support is a team of experts who takes care of your dell printer issue. Our support is capable of dealing with any dell printer glitches. In case you're having issue with you dell printer contact dell Printer Support and get instant help.

  61. bestlaptops12 says:

    bestlaptops12

  62. bestphones90 says:

    bestphones90

  63. bestmobiles44 says:

    bestmobiles44

  64. bestlaptop22 says:

    bestlaptop22

  65. bestlaptopsunder40000 says:

    bestlaptopsunder40000

  66. bestphonesunder15000 says:

    bestphonesunder15000

  67. bestlaptopsunder25000 says:

    bestlaptopsunder25000

  68. bestlaptopsunder20000 says:

    bestlaptopsunder20000

  69. bestphonesunder5000 says:

    bestphonesunder5000

  70. bestphonesunder10000 says:

    bestphonesunder10000

  71. sandeep maheshwari says:

    sandeepmaheshwari

  72. sandeep maheshwari quotes says:

    sandeep maheshwari quotes

  73. AOL Tech Support says:

    AOL Tech Support covers all your AOL mail related issues. The experts at AOL Tech Support are quite proficient in dealing with your issues with ease. For any help related to AOL, contact AOL Tech Support

  74. iphongthuy net says:

    http://iphongthuynet.cd.st/
    http://iphongthuynet.id.st/
    http://iphongthuynet.lo.gs/
    http://iphongthuynet.eklablog.fr/
    http://iphongthuynet.ek.la/
    http://iphongthuynet.eklablog.net/
    http://iphongthuynet.eklablog.com/

    https://www.lonelyplanet.com/profile/iphongthuynet
    https://www.flickr.com/people/iphongthuynet
    https://medium.com/@iphongthuynet
    https://gab.ai/iphongthuynet
    https://disqus.com/by/iphongthuynet
    https://www.amazon.com/gp/profile/amzn1.account.AFGO7U63BCFOJXJMKF36BTWPRUNA
    https://soundcloud.com/iphongthuynet

  75. Assignment Help says:

    Assignment Firm service providing all college students online assignment help. Our experts provide 100% plagiarism free content for college and university students. Thank you.

  76. AOL technical support says:

    Aol tech support number
    Aol tech support number +1800-684-5649 is a customer care helpline number which gives
    you the facility to contact us directly when needed and shootout your problems online
    in just few minutes via online contact.
    https://aolnumbersupport.com/aol-help/

  77. discussion post says:

    Writing a discussion post can be a real challenge, especially if it is the first time you have encountered such an assignment. It might not be easy at first as you might feel discomfort participating in the discussions freely. But everyone can get efficient writing help with this kind of assignment from experienced writers

  78. Mason Kirkby says:

    inotify is my personal favorite, I don;t know about others but i have been using this for a while now. and i recommend this to everyone.

    https://crackmods.com/connectify-hotspot-pro-2019-crack/

  79. hostinger coupon code hosting says:

    This is a great place to get such type of nice post.

  80. Tree Removal In Red Deer says:

    I am very happy to see this post because of it really a nice post.

  81. assignment writing service in adelaide says:

    ABC Assignment Help is an incomparable online assignment help company delivering excellent academic assignments, essays, coursework and reports. Through a team of over 3000 subject experts we ensure individual attention to every student making the assignment help experience completely personalized in nature. With our round the clock services, you can be assured of high grades every time.

  82. AllAssignmenthelp.com reviews says:

    Your article post is decent and valuable information

  83. Gclub says:

    I love this post.

  84. Gclub-royal1688 says:

    Very cool!

  85. Assignment hep says:

    Thanks

  86. Management assignment help says:

    Sample Assignment bestows over the college going students with online assignment help. It is a consultancy possessing academic experts providing a number of subject-specific assignment helps. Marketing assignment help, economics assignment help, MATLAB assignment Help, MySQL assignment help, management assignment help, etc

Leave a Reply