In this tutorial, we’ll look at how we can monitor file changes using Python. We’re going to use Watchdog which is a Python library that offers APIs and shell utilities to monitor file changes using Python. We’ll take a look at how it works, what it can be used for, and provide some insight into how you can use it to monitor file system events in your own applications.

What is Python Watchdog?

The Watchdog is an open-source application that allows you to monitor file changes using Python. Watchdog could be used for a number of common tasks, such as watching for changes in file systems, watching for file systems events, and observing file systems directory. Watchdog operates by registering file system event handlers that fire whenever the corresponding event occurs. Handlers can be registered in response to events on a single file, or on all events on a directory. Users can also provide multiple event handlers and can execute shell scripts when an event is triggered.

Watchdog is a Python library that provides API and shell utilities to monitor file changes using Python. Events include changes to files, changes to the list of files within a directory, and changes to file meta-data such as access control lists. Changes to file metadata are monitored as well including file ownership, filegroup, file flags, and file access control list. 

Prerequisites 

In order to ensure files are present on your computer and performing as they should, it may be important for you to set up a framework for keeping track of their whereabouts and consistency over time. To do this, you will need the following:

Operating system Dependency (row)WindowsLinux Mac OS X/Darwin
XCodeNoNoYes
PyYAMLYesYesYes
arghYesYesYes
argparseYesYesYes
select_backport (Python 2.6)YesYesYes
pathtoolsYesYesYes

Watchdog itself can simply be installed with the following command: 

pip3 install watchdog

Or it can be Installed from the code repository like so: 

git clone --recursive git://github.com/gorakhargosh/watchdog.git
cd watchdog
python3 setup.py install

Implementation

The main building blocks of watchdog are based on two parts the watcher or observer and the event handler. 

Watcher

The Observer class is at the heart of this recipe. It will monitor file changes using Python and the directory passed to it, watching all files as well as any subdirectories that are created. The Watcher object recognizes changes to files in the folder that it’s monitoring and fires an event for your application whenever such a change occurs. If a change occurs anywhere within the monitored directory (or sub-directory), the call method will be called with a hash containing details about the change. 

Handler

The Handler object applies your code to determine what to do with events received from the Watcher. A handler is written by you and you can decide anything you want to do with it such as just viewing it live store it somewhere or automating some tasks to it. It could also be used on websites for security purposes.

Watchdog’s implementation could be done simply in any project by following the method:

  1. Create an instance of the watchdog.observers.Observer thread class. The Observer design pattern works well for creating a thread that acts on events.
  2. Implement a subclass of watchdog.events.FileSystemEventHandler (or just add watchdog.events.LoggingEventHandler)
  3. Set paths to be monitored with the observer instance attaching the event handler.
  4. Start the observer thread and monitor file changes using Python

Note: By default, an watchdog.observers.Observer instance will not monitor sub-directories, only by passing ‘recursive=True’ in the call, it would able to monitor sub-directories

Code

Setting up Watchdog is pretty simple. You just have to first register the file system event handler that you want to use, then wait for events to occur, and finally, act accordingly. For example, if you’d like to be notified every time a directory or file changes its status within your application’s operating system (e.g., becomes open or closed, added or removed), simply register the observer property handler with the directory containing that file or all files in a given directory. The handler will also be triggered if any of the core file attributes – including modified time or directory are updated. You can see the example below: 

import sys
import time
import logging
from watchdog.observers import Observer  #creating an instance of the watchdog.observers.Observer from watchdogs class.
from watchdog.events import LoggingEventHandler  #implementing a subclass of watchdog.events.FileSystemEventHandler which is LoggingEventHandler in our case

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO,
                        format='%(asctime)s - %(message)s')
    path = sys.argv[1] if len(sys.argv) > 1 else '.'
    event_handler = LoggingEventHandler()
    observer = Observer()
    observer.schedule(event_handler, path, recursive=True)  #Scheduling monitoring of a path with the observer instance and event handler. There is 'recursive=True' because only with it enabled, watchdog.observers.Observer can monitor sub-directories
    observer.start()  #for starting the observer thread
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

Here you can see that the path is to the directory where the file is currently located, but you can set it however you want in the following manner: 

path = (r"you files path")

Also, we have to put the r in front of the path to make it a raw string and the program knows exactly where to look for it. After doing that, run the file name in the terminal as the command to start the application.

Results

If the application worked successfully you will see no response but no errors either. You should be seeing an empty terminal right now similar to this:

Now try and run some files where you set the path to be or if you followed the steps exactly as above then it would be where you created this file. Run some files or add some new ones there and should see some results printing out in the terminal, something similar to this:

You can see the time and the message here as we specified above in the code itself, it is currently outputting the location of the files and its name as the message.

Conclusion

That’s it! Here we have created a simple application to monitor the files changes in our system using the python watchdog library. It is a very versatile library that can not only be used for monitoring file changes but also to automate them. It is widely used with corn jobs on supported systems and can also be used on websites for many purposes including security! And these are just a few of them there are tons of features it offers that you can learn from reading from their incredible documentation or get to know their code better from their GitHub. 

Let’s make a quick recap to summarize everything:

First, we installed all the prerequisites required for running the watchdog.

Next, we moved on to explore in-depth the available features and tried out a basic implementation of it. We created a subclass of FileSystemEventHandler and passed it as an input parameter to the Observer object. From there, the input FileSystemEventHandler will trigger events that will invoke corresponding functions when there are modifications to the files or folders in the directory.

Finally, we made some tests via the creation, modification, and deletion of some files.

Here are some useful tutorials that you can read: