Improving mmap/munmap performance on MacOS X

Question!

Running the following C code (a bunch of mmaps and munmaps on a 2GB file) on a MacOS X machine seems to be dramatically slower than on a Linux one.

#define BUFSZ 2000000000
static u_char buf[BUFSZ];
....

// Time 10000 mmaps and munmaps from random offsets for various 
// sizes of mapped chunk.
for (msize = 4096; msize <= 1048576; msize *= 16) {
  fd = open("io_benchmark.dat", O_RDONLY);
  if (fd  < 0 ) die("can't open io_benchmark.dat for reading");
  for (i = 0; i < 10000; i++) {
    // Make sure the block to be mapped doesn't start in the
    // last meg.
    offset = (size_t) random() % (BUFSZ - 1048576);
    mblock = femmap(fd, (off_t)offset, (size_t) msize, PROT_READ, 
                    "test block");
    total = 0;
    for (j = 0; j < msize; j++) {
      total += mblock[j];
    }
    femunmap(mblock, (size_t) msize, "test block");
  }
  printf("Elapsed time to mmap and munmap 10000 blocks of %d kB: %.4f sec\n", 
         msize/1024, (time = time_since_last_call()));

  rslt = close(fd);
  if (fd  < 0 ) die("can't close io_benchmark.dat after reading");
}

Specifically, comparing two machines

CPU     Xeon E3113 dual core @ 3.00GHz           Core 2 Duo @ 2.4GHz dual core
RAM     8GB                                      4GB
Kernel  2.6.18-92.el5PAE SMP i686                MacOS 10.6.4 Snow Leopard
Disk    WD 250GB SATA 16MB cache 7200 RPM EXT3   Hitachi 250GB SATA 5400 RPM, journaled HFS+

Gives the following results

                            Linux    MacOS X
Time for 10000 4kB mmaps    0.0165   682.87
Time for 10000 64kB mmap    0.0170   657.79
Time for 10000 1MB mmaps    0.0217   633.38

Even accounting for the reduced amount of memory, it seems to be an unusual given the file is only half the physical memory. Can anyone point to a change to the code or a configuration change which might improve the performance?

We're tried using reads instead of mmaps, and it does make a substantial difference, but doing that would require a substantial change to an existing code base (and mmap is a lot faster than the read on linux).



Answers

My conclusion to the OS X mmap problem is map the entire file and keep it mapped. If you need to expand the file, map more bytes than you need so that you only need to remap occasionally.

You may need to use 64-bit OS X to make this work.

By : Zan Lynx


Looks like this is "by design". According to this https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/FileSystemAdvancedPT/MappingFilesIntoMemory/MappingFilesIntoMemory.html#//apple_ref/doc/uid/TP40010765-CH2-SW1:

You should not use file mapping in the following situations:

  • You want to read a file sequentially from start to finish only once.

  • The file is several hundred megabytes or more in size. (Mapping large files fills virtual memory space quickly. In addition, your program may not have the available space if it has been running for a while or its memory space is fragmented.)

For large sequential read operations, you are better off disabling disk caching and reading the file into a small memory buffer. See “Cache Files Selectively” for more information.


Here is a code snippet that demonstrates the problem:

off_t       file_target = off_t(3) 
By : Hawkfish


This video can help you solving your question :)
By: admin