October 18, 2006

Clearing Linux Filesystem Read Cache?

Posted in Linux, Questions at 8:24 pm by mj

Anybody know how to clear the Linux filesystem read cache? A good answer would be much appreciated.

I have some performance tests that are being adversely affected by the filesystem’s read cache. If it makes any difference, we’re using ext3. I don’t want to disable the cache, simply clear it between tests.

A chance conversation on my walk to BART this evening got me in mind to simply copy my files between each round of tests. This kind of works. My full test data is ~140GB, but I can get by with ~20GB for tests running during the day. It takes a while to copy, but that’s better than nothing.

Luckily, my data set compresses well (5.5:1 @ 1.5GB/m), so it’s possible to blow away even the full set and restore. I haven’t tried that yet – hopefully the cache is doing a simple “has this inode/block changed recently?” test, and copying the same data back over a cached block still invalidates the cache.

That’s how I’d do it. Actually, I’d implement it using IDirectConnectionToMJsBrain to obtain my intent at every stage. But I won’t fault anybody for overlooking this.

Advertisements

7 Comments »

  1. anon said,

    As of 2.6.16 you can use /proc/sys/vm/drop_caches to clear cache.

    echo 1 > /proc/sys/vm/drop_caches

    Will clear all the fs cache. Do a ‘sync’ before.

    • Ravikumar Wagh said,

      what is difference between “echo 1 > /proc/sys/vm/drop_caches”
      & “echo 2 > /proc/sys/vm/drop_caches” & “echo 3 > /proc/sys/vm/drop_caches”
      and also why sync is required.

      • Ian Macintosh said,

        Ravi, you did not read Sandor’s comment from Nov 25, 2008 nor mine from Oct 19, 2012 which answers the sync question.

        For the details on the difference between 1, 2 & 3 see the documentation on drop_caches.

        https://www.kernel.org/doc/Documentation/sysctl/vm.txt

  2. Andrew Z said,

    Anon’s common seems dangerous. Here’s another option
    http://wiki.services.openoffice.org/wiki/Cold-start-simulator

  3. jm said,

    You can create a large file which use all filesystem cache and then remove it :
    dd if=/dev/zero of=file bs=1024 count=6000000
    rm file

    In my case I create a 6Go file, it depends on system configuration :
    more /proc/meminfo

  4. Sandor said,

    Writing to /proc/sys/vm/drop_caches should not be dangerous. The reason for doing a sync first is so more caches can get dropped.

  5. Ian Macintosh said,

    Writing to /proc/sys/vm/drop_caches is *NOT* dangerous.

    It only drops free cache entries. Dirty cache entries are not free, and are not dropped.

    If you actually want to free all cache, then ‘sync’ first, so that the maximum number of buffers are actually dropped. But it is not at all a requirement of /proc/sys/vm/drop_caches that you do so.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: