File and Data Ingest

BEFORE you begin importing files or data, read the section at the bottom of this article.

Import tools make it easy to transfer your assets and metadata into NetX.

In this article Ask your team
Learn about the different methods of importing assets and metadata. Discover how to prepare your files and data for import and ensure all required configurations are in place prior to migrating your content.

5 things to know about an initial ingest of files and data

Creating assets in NetX is more than just copying files from one place to another.
NetX generates derivative files (thumbnails, previews, zooms, pages, keyframes, etc.) at the time of import, so a large import might take longer than you expect.
A small complex file can take longer to ingest than a large simple file.
Complex files such as videos, layered TIFFs, multi-page PDF, and InDesign documents can take some time to process.
Metadata is indexed by the search engine.
System attributes such as filenames, folder paths, and some embedded metadata are indexed at the time of file import. Custom attributes are indexed at the time of metadata upload or when edited in NetX.
Metadata from a previous DAM can be imported via .csv data upload.
All data migrating from another system must be formatted correctly and align with your NetX attributes. If you need to migrate very large amounts of data, our professional services team can help wrangle your existing metadata.
NetX Professional Services Team can help expedite your initial file ingest.
From shipping a hard drive and letting us manage your ingest, to deploying extra application servers to share the ingest load, ask your NetX Representative about the right approach for your organization. 

Best practices

Plan ahead.
Assign sufficient resources to prepare and deliver files and data. Migration preparation often takes a lot longer than you imagine. Prioritize popular assets and consider breaking up large or complex import projects into smaller batches.

Let it go.
Take the opportunity to filter out unnecessary files for a clean start in NetX. In many cases, storing only the highest resolution versions is sufficient as derivatives can be created on-demand instead of being stored. Plan carefully whether or not to store raw materials in NetX, such as unretouched photos or unedited HD video. NetX should store files that are of value to your organization, so resist storing everything!

Filenames are your friends.
Filenames are indexed by the search engine — take advantage of this fact. Descriptive, intuitive, consistent, and concise filename conventions are best. Avoid illegal characters and excessively long filenames (technical limit is 255 characters, best practice is less than 25 characters). Case and spelling matter — image.JPG, image.jpg, and image.jpeg are treated as distinct files in NetX. Unique filenames prevent collisions when adding assets to folders or uploading metadata via .csv.

Cataloging is better in bulk.
Attribute values can easily be applied to batches of assets at the time of import using the browser upload tool or the NetX I/O app. 

Import your way.
Manage your own file and data ingest via the browser uploader and .csv import tools or engage our professional services team to handle things for you.   


How-to articles

File import Data import

Tips and tricks

Login as a generic "system" Admin user instead of your own user account to avoid accidental import interruptions. Using a generic user (such as netx_admin or import_admin) can be helpful for statistics and to distinguish files in the initial import from subsequent imports. 

For initial ingests via NetX I/O or a shipped hard drive, subfolders will be automatically created during the ingest. Only the top-level structure needs to be created ahead of time in NetX. 

For initial ingests via the browser, all subfolders need to be created before uploading files.

To increase indexing efficiency, all folder permissions should be configured prior to your initial import. Add all folders and sub-folders to which permissions apply, create user groups, then configure permissions.

If a filename collision occurs, subsequent assets will be appended -1, -2, etc. which may impact data imports or other workflows dependent on accurate filenames. See the section on duplicate filenames in the Uploading Assets article for details.

Duplicate Asset Detection can be configured if you have a large number of duplicate assets that need to be weeded out. See point 5 in the section below if you wish to use this feature.


*BEFORE YOU IMPORT ASSETS OR METADATA*

1. Review our File Ingest Policy.

2. Provide these details to your Onboarding Specialist (estimates are fine):

  • total size in GB/TB of the import
  • total number of files
  • average file size
  • percentage of each file type
    • (e.g. 50% images, 40% video, 10% multi-page documents)

3. Ensure the following essential configurations are in place:

  • all top-level folders
  • any subfolders that have permissions
  • user groups
  • permissions
  • all properties that alter how files or data are processed
  • properties to optimize an initial ingest

4. If your site uses any optional configurations, these should be in place prior to import, along with all associated attribute fields, vocabularies, folders, and permissions:

  • attribute profiles
  • embedded metadata mappings triggered on import
  • autotask workflows triggered on import
  • autotasks that generate views or derivatives
  • indexing document content for basic keyword search
  • embedded color profiles for proxies
  • advanced configurations such as:
    • .jar files for custom jobs
    • MediaRich Zooms
    • InDesign Server
    • Hydra configuration
  • any other configurations that are impacted by the import of files or data

5. If you're using Duplicate Asset Detection:

  • turn on MD5 Checksums prior to the initial ingest, but DO NOT turn on the LinkDuplicatesJob. This ensures the asset count in NetX can be verified against the original ingest manifest.
  • once the ingest completes and has been verified, export the following attributes for all assets: Asset Id, File, Folder Path, Checksum. Review these files before proceeding with Duplicate Asset Detection. Identify duplicate checksums using Excel and then Upload Selection to NetX
  • turn on the LinkDuplicatesJob and then initiate an index-only resync on the Selection to remove any duplicates (see Resyncing Assets). 
  • filenames are ignored by this system, so identical files with different filenames will still be detected as duplicates. Conversely, non-identical files with the same filenames will not be caught by Duplicate Asset Detection.

Up next...

Launching your DAM

Was this article helpful?
0 out of 0 found this helpful