Dynamic Libraries outside of Standard dyld search paths

Fun side effects of building tools from source

I use the CommonMark library that Odin ships with vendor support for in the generation of the HTML pages for this blog. On my system, however, that means installing CommonMark. I'm a big proponent of building things from source. However, one of my biggest gripes with this is most tools install to usr/bin or usr/local/bin and this requires sudo access. I have a directory in my $HOME directory called .bins that contains all the tools I've built and installed from source over the years using the .configure script. I find this works a lot better than poking around in usr/bin. For CommonMark the directory looks like this, ~/.bins/.cmark/lib, ~/.bins/.cmark/bin, and ~/.bins/.cmark/share

A problem with this however, is that when Odin builds and links the object file it goes something like this. First, ld looks for the cmark/lib folder so it can add a reference to the .dylib or .so library depending on how it was installed. However, it only knows to look in usr/bin or usr/local/bin. From the man pages for ld.

ld maintains a list of directories to search for a library or framework to use. The default library search path is /usr/lib then /usr/local/lib. The -L option will add a new library search path.

That solves the first problem. Odin exposes a clean way to pass extra linker flags, with the -extra-linker-flags argument. Simply pass -extra-linker-flags="-L$CMARK_LIB_PATH" and the linker will be able to reference it. Once this works, however, we run into the next problem of loading the dynamic library.

I tried running the binary, and got this error. dyld[73494]: Library not loaded: @rpath/libcmark.0.31.2.dylib. A little further down, Reason: tried: '/Users/pn320/code/theatres-of-anarchy/libcmark.0.31.2.dylib'. It seems to be looking for the shared library in the same folder as the binary.

You can inspect the generated binary using otool -L $BINARY_NAME and this shows you what the binary will ask dyld when loading the reference. In most cases, it should look something like this, @rpath/libcmark.0.31.2.dylib. @rpath is just a placeholder for "look at the list of runtime paths embedded in the executable".

You can fix this in two ways, either edit the binary using install_name_tool with the -change flag to set the actual path of the shared library or, and this is my preferred method of doing it, you can tell dyld exactly where else to look with DYLD_FALLBACK_LIBRARY_PATH variable.