NetX and IIIF

April 2018

As a company, NetX has a core commitment to open standards and to make our product as interoperable as possible. I wrote about this commitment back in 2011 with a post titled “
Bigger Than DAM” (https://www.netx.net/bigger-than-dam); there, I argued that DAMs should be an open, standards-based platform for driving a wide variety of enterprise applications.

Since that time, NetX has also committed to serving the DAM requirements of cultural heritage organizations, especially Museums, Libraries, Foundations, and Universities. So when NetX received Sheila Rabun’s email back in May 2017 — “Museum community letter to DAMS vendors, (sent on behalf of the museum’s community) regarding the International Image Interoperability Framework (IIIF)” — we were excited to meet its implicit challenge: NetX should embrace and support IIIF.

We started by reading the IIIF spec; and shortly after, our engineers dug into the APIs. The question of “How would we implement this?”, quickly turned into “What are the use cases to guide us?”. These questions led us to start talking to our customers (and potential customers) about how they would use NetX in conjunction with IIIF. To date, these conversations lead me to believe that this is still largely an open question (more on this later). Furthermore, not only is the IIIF spec itself evolving rapidly, supporting software implementations (Mirador and many others) are also changing rapidly.

This presents a challenge to a software vendor like NetX. I referenced my “Bigger Than DAM” post deliberately, because at that time, it held up (the now defunct) CMIS standard as the “new way forward”. From my vantage, IIIF certainly will be approached with caution because of NetX’s earlier experiences with projects that failed — CMIS is one of many. This is not to sound any alarm with IIIF; in fact, the rapid pace of change within the IIIF spec and supporting open source libraries is incredibly positive — showing energy and commitment behind IIIF. I mention my caution to help frame NetX’s approach to supporting IIIF: we are doing so carefully, methodically, and deliberately in stages.

Working diligently since we received the Museum community letter to DAMS vendors, we are now poised to offer an initial prototype of NetX support for IIIF.

Developing the Prototype

We started thinking boldly: integrating a Mirador viewer into NetX. This would solve a secondary issue we hoped to resolve: providing a markup and approval function within NetX. This led us down three distinct technological tracks: IIIF manifests, IIIF zoom servers, and IIIF annotations.

IIIF manifests are the foundation of any support. So that was the easy place to begin. This component is mostly just an exercise in translating NetX data — thumbnails, zooms, and metadata — into an IIIF format.

For zooms, we have a long history of experience with the various underlying technologies, from Pyramid TIFFs and “Zoomify” tiles, to ImageMagick and various other imaging libraries. However, we decided to integrate with Cantaloupe as it solved many IIIF integration issues out-of-the-box. It sports a very flexible plugin architecture, allowing for various imaging engines to be activated, including ImageMagick, and even Kakadu (which I’ll return to below).

Annotations turned out to be a pretty big bite to chew. It finally led us to rethink any direct integration with Mirador. The IIIF Annotation API is really just a reference to the W3C spec, and unfortunately it — like the WebDAV RFC and many others — leaves many details undefined. In fact, we found no off-the-shelf annotation library and ended up writing our own. We built it deliberately so that it’s compatible with the W3C spec, but — as there’s little detail on how commenting should be defined — we simply designed the system to provide the best user experience. This includes venturing into the world of WebSockets to allow real-time collaboration.

Despite the fact we decided not to include Mirador within the NetX system directly, we are using Mirador as a “reference implementation” of an IIIF viewer to ensure NetX IIIF functions remain compatible moving forward. In the prototype, IIIF manifests will be available for loading into a viewer like Mirador, providing NetX metadata along with zooming (via Cantaloupe). Effectively, manifest URLs have authenticated session requirements (see more on Authentication issues below), and therefore NetX manifests necessarily expire after a time. This session requirement is currently required to be compatible with NetX’s security model — all access in NetX requires an authenticated user.

We are considering augmenting this system in a future iteration with NetX’s “token” access, allowing for permanent and semi-permanent manifest URLs, but we need solid use cases (more on this below).

While our vision is to integrate our new Review and Approval feature with our overall IIIF support,  for the time being, annotations are separate from IIIF.

Manifests and the underlying zoom functionality provide the real first step towards IIIF support. NetX’s AutoTask workflow engine activates this function. Doing so then instructs NetX to produce a Pyramid TIFF (which is then fed to Cantaloupe). This allows each implementation to define which assets are exposed via IIIF — folder designations, metadata and other criteria can be defined. More importantly, this defines how much additional storage is required for these TIFFs (stored in NetX as Asset Views).

Pyramid 2000

When we first embarked on this project, JPEG 2000 seemed to be the obvious choice to drive IIIF zooms — the TIFF format seems like an anachronistic dinosaur when compared to the more modern JPEG 2000 format. However, without a commercial server like Kakadu, in our tests, open source servers like ImageMagick and OpenJPEG just weren’t as performant as Pyramid TIFFs. However, as NetX can easily be reconfigured to produce JP2 files instead of TIFFs, and similarly, Cantaloupe can also be reconfigured to use OpenJPEG or Kakadu — we hope to support either option in future iterations. At this first step, we’re officially supporting Pyramid TIFFs.

This is the Beginning

The most significant point I want to make in this post is that NetX’s IIIF prototype is only a foundational beginning. I wanted to take a deliberate step forward, but to proceed cautiously. We are in need of real-world use cases to help us reduce the myriad potential roads we might wander down. We look forward to working not only with the IIIF community, but also with our customers (and potential customers) on how we should proceed from here.

We have many questions (and concerns), including:

  • First and foremost, we need to reconcile the “openness” of IIIF — sharing manifests publicly, and between institutions — with the security requirements (the “closedness”) of NetX. For example, how can you annotate an asset without logging in, so we know who is doing the markup and commenting? Is there a desire for NetX to support both authenticated and anonymous access?

  • In our initial implementation, IIIF manifests are available on an asset by asset basis. The reason NetX doesn’t yet support collections for IIIF goes back to the issue of authentication: who, what, and how — exactly? For example, should designated folder structures be deemed “public” for IIIF support? Or, what about metadata-defined collections (using NetX saved searches)?

  • Do organizations want to publish asset data (file and metadata) to an external IIIF server? If so, how might that work? And would that replace a direct integration (as we’re starting to provide), or would organizations use both approaches?

  • What about storage issues of the zoom files? Whether they’re Pyramid TIFFs, JP2s, or even zoom tiles — pre-generating the zoom file is a necessary evil considering the user experience, and the storage requirements generally are not trivial. In our experience and tests, asset source files are typically 100MB+ (and often 250MBs); these can take close to a minute to generate either a Pyramid TIFF or JP2. Today, we think that’s too long for a user to wait to see zoom functions. But if that were acceptable (or if we can reduce that wait), then we’d prefer to build those files on-demand to reduce the significant storage bloat these files cause (in some cases, the generated zoom file can be larger than the asset original).

  • How should a vendor like NetX get involved? In the case of annotations, we are extending the W3C spec to provide a better user experience; but clearly there’s a danger if these choices are not reincorporated back into the spec. Again, NetX’s core mission is to remain open. On the flip side, we are also challenged by how quickly various open source IIIF libraries are changing (the manifest library we chose last year is already deprecated in favor of a complete rewrite). This rapid evolution is both a benefit and a hazard; for a vendor, constant refactoring is expensive.

We initiated this project to show our commitment to this endeavor, but also to begin a conversation. The next step is developing a common set of use cases, so we can proceed with improving our IIIF integration. We look forward to talking with you!

Jason Wehling

President

NetX

 

Was this article helpful?
2 out of 2 found this helpful