External library woes

Hello

I’m struggling a bit with getting the linker going in a nodeset as far as I understand.

The nodes compile fine without complaint from both Clang and GCC until I run them in Vuo. Then i get an error:

Undefined symbols for architecture x86_64: “_ffclos”, referenced from: _display_fits_image in composite…

… clang: error: linker command failed with exit code 1 (use -v to see invocation)

Dependencies are set in the description, locations are added in CMake, but I guess I’m doing something wrong or have put something in the wrong place.

In my CMake file, I have this:

...
COMMAND ${VUO_FRAMEWORK}/../vuo-compile
    "${PROJECT_SOURCE_DIR}/${source}"
    --target ${arch}-apple-macos10.12.0
    --output ${bitcode}
    --library-search-path /usr/local/Cellar/cfitsio/4.2.0/lib      //where the library resides
OUTPUT ${bitcode}

(there is no comment in the real CMake.txt)

My metadata contains:

“dependencies”: [“cfitsio.dylib”, …] (path is /usr/local/Cellar/cfitsio/4.2.0/lib/libcfitsio.dylib)

What might I do to fix this issue?

1 Like

If all you want to do is run compositions that use cfitsio nodes on your own computer, then the fix is simple — just copy libcfitsio.dylib to the User Modules folder alongside your node classes. Attached is an example project that demonstrates.

When you run a composition in the Vuo editor, Vuo compiles and links the composition. As you noted, it’s the linking step that fails (undefined symbols). The Vuo editor has certain predetermined places where it looks for libraries, including the User Modules folder.

The --library-search-path argument that you mentioned comes into play when you use the vuo-link command-line tool to link a composition. If you were to create a composition that uses your cfitsio node, and then compile and link it on the command line, the commands would look like this:

/Library/Developer/Vuo/framework/vuo-compile --header-search-path /usr/local/Cellar/cfitsio/4.2.0/include ~/Desktop/TestCfitsio.vuo    #compile
/Library/Developer/Vuo/framework/vuo-link --library-search-path /usr/local/Cellar/cfitsio/4.2.0/lib ~/Desktop/TestCfitsio.bc    #link
~/Desktop/TestCfitsio    #run

If you plan to use cfitsio nodes/compositions on other computers, then it gets more complicated. If you have control over the other computer and can install libcfitsio and rebuild the node classes on that computer, that would probably be the simplest option. When building stuff on one computer and running it on another, you get into multi-architecture builds (Intel + Apple Silicon), dylib install names, and macOS Gatekeeper. I can try to help if you start down that route and get stuck.

example-cfitsio.zip (3.29 KB)

1 Like

Thanks Jaymie! Having an easily transferrable solution would be the ideal solution, as in having it as a nodeset to just pop over to different computers. There might be a middle ground if its only a matter of the library being installed to the same location that requires rebuilding. I’m using Homebrew to install the libraries, so both cfitsio and eventual other libraries installed using that should always reside at /usr/local/Cellar if not changed by the end user.

Cfitsio is at least mulit-platform and supported on Apple Silicon, so hopefully it isn’t the worst of libraries to try to include

1 Like

Yeah, installing the library on the other computer with Homebrew might simplify things. (Though keep in mind that Homebrew installs to /opt/homebrew/Cellar by default on ARM instead of /usr/local/Cellar.)

In any case, I look forward to seeing your progress and how you use the nodes, if you’d like to share :)

Aha! Could the /usr/***/Cellar be dependant on architecture in the cmake file perhaps?

I also have to admit to using some performance enhancers for this experiment (chatGPT), but as it compiles fine, and it is the linker that is complaining, I think it has been coaxed into providing something that should run at least.

Using the example you provided, I got a little bit farther. I did not get an error when running your example node, but this one still spits out a linker error:

ld: warning: passed two min versions (10.10.0, 10.12) for platform macOS. Using 10.12.
Undefined symbols for architecture x86_64:
“_fits_getimg_param”, referenced from:
_display_fits_image in composite-Aq6gm3-95525c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

C code for the node:

/**
 * @file
 * MM.image.fetchFits node implementation.
 *
 * @copyright Copyright © 2012–2022 Magneson.
 * This code may be modified and distributed under the terms of the MIT License.
 * For more information, see https://vuo.org/license.
 */

#include "node.h"
#include "fitsio.h"
#include "fitsio2.h"
#include <OpenGL/gl.h>


VuoModuleMetadata({
					 "title" : "Fetch FITS",
					 "description" : "Fetches a FITS file as a Vuo Image",
					 "keywords" : [ ],
					 "version" : "1.0.0",
					 "dependencies" : [ "cfitsio.dylib" ],
				 });

VuoImage display_fits_image(const char *filename) {
    fitsfile *fptr;
    int status = 0;
    int bitpix, naxis;
    long naxes[FITS_MAX_DIM];
    unsigned char *pixels;
    int x, y;

    // Open FITS file.
    fits_open_file(&fptr, filename, READONLY, &status);

    // Get the image dimensions and type.
    fits_get_img_param(fptr, FITS_MAX_DIM, &bitpix, &naxis, naxes, &status);

    // Allocate memory for the pixel data.
    pixels = (unsigned char *) malloc(naxes[0] * naxes[1] * sizeof(unsigned char));

    // Read the image data.
    long fpixel[FITS_MAX_DIM];
    fpixel[0] = 1;
    fpixel[1] = 1;
    long nelements = naxes[0] * naxes[1];
    for (int i = 2; i <= naxis; i++) {
        nelements *= naxes[i-1];
        fpixel[i] = 1;
    }
    fits_read_pix(fptr, TBYTE, fpixel, nelements, NULL, pixels, NULL, &status);

    // Close the FITS file.
    fits_close_file(fptr, &status);

    // Create a GL texture from the pixel data.
    unsigned int textureName;
    glGenTextures(1, &textureName);
    glBindTexture(GL_TEXTURE_2D, textureName);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, naxes[0], naxes[1], 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, pixels);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

    // Create VuoImage from GL texture.
    VuoImage image = VuoImage_make(textureName, GL_LUMINANCE, naxes[0], naxes[1]);

    // Free the pixel data.
    free(pixels);

    // Return VuoImage.
    return image;
}

void nodeEvent
(
		VuoInputData(VuoText) url,
		VuoOutputData(VuoImage) fitsImage
)
{
	*fitsImage = display_fits_image(url);
}

Relevant part of CMakeLists.txt:

set(compiledNodes "")
foreach (source ${sources})
	set(compiledNode "")
	set(bitcodeParts "")
	list(LENGTH CMAKE_OSX_ARCHITECTURES archCount)
	foreach (arch ${CMAKE_OSX_ARCHITECTURES})
		get_filename_component(bitcode ${source} NAME_WLE)
		set(compiledNode "${bitcode}.vuonode")
		if (archCount EQUAL 1)
			set(bitcode "${bitcode}.vuonode")
		else()
			set(bitcode "${bitcode}-${arch}.vuonode")
		endif()
	
		add_custom_command(
			DEPENDS "${PROJECT_SOURCE_DIR}/${source}"
			COMMENT "Compiling node class (${arch})"
			COMMAND ${VUO_FRAMEWORK}/../vuo-compile
				"${PROJECT_SOURCE_DIR}/${source}"
				--target ${arch}-apple-macosx10.12.0
				--output ${bitcode}
				--header-search-path /usr/local/Cellar/cfitsio/4.2.0/include  # for #include "fitsio.h"
				OUTPUT ${bitcode}
		)
	
		if (archCount GREATER 1)
			list(APPEND bitcodeParts ${bitcode})
		endif()
	endforeach()
	
	if (archCount GREATER 1)
		add_custom_command(
			DEPENDS ${bitcodeParts}
			COMMENT "Merging into multi-architecture file"
			COMMAND lipo -create ${bitcodeParts} -output ${compiledNode}
			OUTPUT ${compiledNode}
		)
	endif()

	list(APPEND compiledNodes ${compiledNode})
endforeach()

#Name your nodeset
set(nodeset MM.image.vuonode)
add_custom_command(
	DEPENDS ${compiledNodes}
	COMMENT "Packaging node set and copying to User Modules folder"
	COMMAND zip --quiet --junk-paths ${nodeset}
		${compiledNodes}
	COMMAND cp ${nodeset} ${userModules}
	OUTPUT ${nodeset}
)

set(installedDylib ${userModules}/libcfitsio.dylib)
add_custom_command(
	COMMENT "Copying libcfitsio.dylib to User Modules folder"
	COMMAND cp /usr/local/Cellar/cfitsio/4.2.0/lib/libcfitsio.dylib ${installedDylib}  
	OUTPUT ${installedDylib}
)


add_custom_target(node ALL
	DEPENDS ${installedNode} ${installedDylib}
)

With the code you posted, I tried to reproduce the “undefined symbols” error, but it didn’t happen for me. When I built the CMake project and used the resulting node class in a composition, I was able to run the composition without any linker errors.

Are you still building the project and running the composition on the same computer? Or does this only happen when running on a different computer?

One thing I noticed when building your project was that the last part of your CMakeLists.txt —

add_custom_target(node ALL
    DEPENDS ${installedNode} ${installedDylib}
)

— refers to ${installedNode} instead of ${nodeset}. So when you modify your node class code and run make, it may not be updating the node set installed in User Modules. That wouldn’t directly cause the linker error, but it could lead to confusing results in general.

Could the /usr/***/Cellar be dependant on architecture in the cmake file perhaps?

Yeah, something like:

execute_process(COMMAND /usr/bin/uname -m OUTPUT_VARIABLE HOST_ARCH OUTPUT_STRIP_TRAILING_WHITESPACE)
if (${HOST_ARCH} STREQUAL "x86_64")
    set(path "/usr/local/Cellar/cfitsio/4.2.0")
else()
    set(path "/opt/homebrew/Cellar/cfitsio/4.2.0")
endif()