We just went through a render farm tech support call with a customer using Maya / Mental Ray on RenderFarmer 1.1, and I thought this information might be useful to others. The problem was that Mental Ray likes to use /usr/tmp for its local frame buffer, but /usr/tmp is shared by all nodes because of the shared root filesystem. So not only is it slow to access because it's across the network, but the nodes are writing all over each others temp files.
First we tried a bind mount that hooks /usr/tmp into /tmp, which was mounted on a local ramdisk. The access speed was great, but processing time was very slow. Why? The ramdisk was tiny, just 10 MB by default. Nowhere near enough for Mental Ray scratch space.
Solution two was to remove the symlink on each node to /usr/tmp, then create a local /usr/tmp (inside the rootfs, which is also a ramdisk). Presto! We now have a /usr/tmp that is not limited to an arbitrary size, and because it's in memory, it's lightning fast.
| Write speeds observed on RenderFarmer 1.1 cluster using gigabit ethernet : | |
| Compute Node --> remote filesystem | 70 MB/s |
| Master Server --> local disk filesystem | 720 MB/s |
| Compute Node --> local ramdisk /usr/tmp | 2900 MB/s |