NetX I/O 4 User Guide

The NetX I/O app provides a robust file transfer and synchronization tool that removes file management barriers and enables teams to work directly from the desktop. With NetX I/O, you can:

• Transfer files to and from a remote NetX instance.
• Synchronize a local file server or desktop with NetX instances (remote or local) and ensure the latest version of your files are always available from anywhere.
• Let creators work directly on the file system, but get the benefits of fully managed assets in NetX.
• Make NetX a pervasive part of your workflow, without accessing the application itself.
• Preserve a single source of assets, locally or remotely, to keep libraries up-to-date. 
• Specify system memory usage, syncing frequency, deletion limit, and watched folders that map to NetX.

There are six operating Modes and an Upload action available through the NetX I/O app. The Upload action in NetX I/O essentially mirrors the upload action in your NetX instance with a slightly different interface. The Modes are used to performed advanced file management tasks that can be scheduled to run at regular intervals if desired.

Installation

See NetX I/O 4 Installation Guide for instructions on how to download and install NetX I/O. If you wish to install the CLI tool, see NetX I/O 4 CLI.

All about modes

The various Modes that can be configured in NetX I/O all manage files and folder structures differently. Details on the function of each Mode are below.

Scenario	Mode
You want to keep a mirror of files and folders on a local file system, but you don't want anything to get deleted, ever.	Merge
You want to keep a mirror of files and folders on a local file system, and you want any changes on either side (NetX and locally) to reflect at the other location.	Sync
You have a bunch of folders and files you need to import into NetX.	Import
You want to download a section of NetX, including folders and files — perhaps for a project, or maybe taking a backup.	Export
You need to periodically upload files to specific folders in NetX regularly; but once uploaded, you don't need those files to remain locally.	Watched folder

Import mode

NetX I/O provides for straightforward importing of the content of a folder on your local directory structure. Import mode simply uploads the content of a local directory structure (the specified folder's subfolders and files) into NetX. A folder is chosen in both NetX and your local environment and the subfolders and files in your local folder are copied/uploaded into the NetX folder. Files can be imported "blind" (where NetX does not compare incoming files with those already in NetX), or it can be run in the default mode which skips reimporting files that already exist in NetX.

Export mode

Export mode moves in the opposite direction of Import mode. A folder is chosen in NetX and another in your local directory structure. Any subfolders and files in the NetX folder are then downloaded to a chosen folder in your local environment.

Watched folder mode

In Watched Folder mode, NetX I/O transfers the files and subfolders of a chosen local directory to NetX. It's a one-way street: when files are copied into the Watched Folder or any of its subfolders, those files and folders are transferred to NetX on a configured schedule. This system deletes files that have been uploaded to NetX from the local directory but leaves the folder structure in place. This is done deliberately so users can more easily drop files into a structure that already exists in NetX instead of having to create those folders each time. Watched Folder mode may be set to "static" source files in the configuration process. This bypasses the initial check that files and folders are not in the process of changing (being saved, moved, or renamed). Watched Folder mode also supports the "blind" option (see above).

Merge mode

Merge mode copies all differences between the chosen local folder and the NetX folder. This propagates all additions to their respective locations. While in Merge mode, no files are deleted from either location. However, if there are conflicts, then NetX is the authority on the correct file. If you have a NetX folder structure and you merge that with a local directory and that local directory already has folders and files, NetX asset files will replace the conflicting local files in any cases where the paths conflict.  As an example, if you have Folder A in NetX Merged with Folder A on your local system and they both contain document.pdf but the content of those documents differ, document.pdf in the local environment will be replaced with document.pdf from your NetX instance. Conflicting local files will be moved to the Lost and Found folder.

Sync mode

Sync mode synchronizes files between NetX and the local folder. For syncing, NetX I/O uses state maps — maps of the state of the local file system and NetX. State maps are stored in the NetX database. Until there is a state map, NetX I/O will initially run in Merge mode, which will automatically build these maps.

In Sync mode, all files are replicated between the local directory and the chosen NetX folder. Added or deleted files and folders will be mirrored to the "other side" with each run. The local state map retains the last known "state" and uses that to determine what tasks are to be done: the map is compared to the NetX folder and the local file system to determine what has changed in each location. Adds (imports) are executed before any files are deleted. Like Merge mode, NetX is the authority on the correct file if any files conflict.

The state map stored in the database is linked to the sync by the clientId. The clientId is composed of the user identification, the folder paths in both NetX and the local environment, and other items. When a mismatch occurs for any reason (logging into the NetX I/O application as a different user, a change in folder name, etc.) the stored state map is disregarded and the Sync mode goes back to the beginning performing a merge and recreating the state map. 

Known limitations and restrictions

Below is a list of known limitations, restrictions, and other good-to-know details about NetX I/O:

Limitation or restriction	Details
Only 'online' or S3-stored Assets	NetX I/O only works with Online Assets — those stored in the Online Repository, or S3-based assets. This function does not work with Offline storage locations. It will simply not sync down (NetX -> Local) any assets that are not online; UNKNOWNs, and other statuses will be ignored. Therefore, this shouldn't be used in conjunction with other Storage Locations because of the potential for conflicts. What if you have an UNKNOWN asset and you add a file for that into the local folder? That would likely version that UNKNOWN asset. Therefore it's not supported.
Local file renaming/moving AND file updates	Moving files around locally is supported. Renaming files locally is supported. Updating files locally is supported. But, combining either a move or a rename with a modification (changing its internal bits) to a local file — this combination is not supported. In this case, the NetX Asset will be deleted; and the locally modified file will be reimported. This is not ideal as any server-side metadata will likely be lost. If you need to do both actions, please be careful to perform updates in a separate sync execution from any file renaming or moving. It's worth noting the in the reverse scenario — where you update a file and move it in NetX — the file will be refetched from the server regardless.
NetX must be available	The remote NetX server must be available during the running of NetX I/O. If NetX is not available, NetX I/O will halt. However, NetX I/O is capable of handling a restart of NetX as long as the restart completes between the process cycles. For example, if you have it set to loop every 15 minutes, NetX can be restarted during those 15-minute pause windows and NetX I/O will happily re-authenticate. But if it can't authenticate, it will halt.
Versions must match	The versions between NetX I/O and the NetX server must match — both major and minor versions. This tool retains that restriction as well. It will enforce the major and minor (but not the patch) versions. So if you are running 8.11, then the jar for NetX IO must also be 8.11 or it will halt at startup and report the error.
Excessive asset deletions are deliberately blocked	To mitigate against the accidental deletion of a large section of NetX assets by a user error, NetX I/O has a default deletion limit of 100. If — in a single sync execution run — it finds that NetX I/O will delete more than the deletion limit in NetX, then NetX I/O assumes a mistake; it will then email the admin, and halt. This is a deliberate insurance policy. Please refer to the CLI article, and the "deleteLimit" parameter, for more details on changing (and/or removing) this limit to suit your needs.
Don't log out	It's better to create a separate account for your NetX I/O configured user; if you were to log in to the NetX server, and then log out with that user, it could disrupt the NetX IO process.
Folder link limitation	You can sync assets that live in multiple folders in NetX; however, you can't sync with a NetX folder that contains assets with multiple folder links within the sync root. That is, assume you have an asset in folder A and folder B; you can sync with either folder and it works. However, if you have folder A, and you have sub-folders B and C, and you have an asset that lives in B and C — in this scenario, you can't sync with folder A. This is because the multiple paths for a single asset are not supported. In this situation, NetX will log an ERROR (with an email), and exit.
Illegal characters	NetX does not accept the following characters for files: "/", "\\", "..", "+", "\", ":", "*", "|", ">", "<","?","\n","\r" Please use the "Replacement Character" option to automatically fix such issues with files and folders on your local sync point.
No backward patching	If features are added or bugs are fixed in future versions of NetX I/O, those will not be patched to older versions. Instead, please upgrade to a newer version to obtain those bug fixes and new functions.
You can't sync everything	You can't pass in the root sync folder path as "/"; folder ID = 1 is not allowed (it requires the underlying folderId > 1). Therefore, you can't sync everything; this is by design. You can sync a top level folder, but not sync the "top level" itself. Please note: Watched Folder mode is the exception; when running in Watched Folder mode, you can set the NetX folder to the top level ("/"). As a rule of thumb: don't try syncing more than 50,000 assets per configuration.
Only Admins for sync and merge modes, Producers for the other modes	If you're running NetX I/O in 'sync' or 'merge' mode, you must configure it to run with an Administrator-level user. For other modes, a Producer-level user (or higher) is required.
Only once per minute, max	You can't run the tool more than once per minute. If you do, the tool will issue a WARN level log, and will pause until the one-minute restriction is met.

Installation

The NetX I/O application can be installed on Windows, Mac OS, and Linux. For instructions on how to install I/O on your platform of choice, see the NetX I/O 4 Installation Guide.

Connecting NetX I/O to your NetX instance

To connect NetX I/O to your NetX instance, navigate to the NetX I/O Account tab. There you will enter the URL of your NetX instance: 

The advanced options allow you to:

• Set the HTTP Timeout (how long before NetX I/O checks in with NetX while not performing a task). Default is 3 minutes.
• Determine whether to use in-memory maps for syncing. This is the faster option, but for very large data sets the memory requirements can be quite large. In such cases, you may want to disable in-memory maps which forces NetX I/O to use the server database to compute map deltas when syncing.
• If you use an HTTP/S proxy to access the NetX server from your local LAN, you can enter such details into the host and port inputs.
• Set the NetX I/O logs to Debug mode (sometimes requested by support for troubleshooting).
  

 

The next view will depend on whether the NetX server has SSO enabled.

Local account login

If you don't have any SSO login options, then you will be presented with a username and password input:

SSO login

If you have SSO login options available, then the next view will instead present those options:

Clicking one of the available SSO option buttons will launch a popup displaying your SSO authentication page. For example:

Once you successfully authenticate there, NetX I/O will obtain a valid session key for interacting with the remote NetX server.

Uploading via I/O

The Upload tab in I/O allows you to drag and drop files and folders into the app and have them moved to the designated folder in NetX. While the instructions below are given in a specific order, these steps (except clicking the Upload button) can be performed in any order.

1. Select the NetX folder that you would to upload the folders and/or files into by clicking the Browse button and selecting the folder from the Folder Tree. You may not select the Top Level folder.

   

   

2. Drag the folders and files to be uploaded into the Upload Gallery. Once they are placed, their icons will indicate whether or not they are a file or a folder. While I/O will upload complete folder structures and their content, only the top level folder dragged in will be shown in the Upload Gallery. Using the Clear button beneath the Upload Gallery will clear both the folder selection and all files and folders dragged to the gallery. It does not clear attributes. 
   
3. Enter Attribute Data.
   1. Click the Attributes button at the bottom of the Upload Gallery. If there are mandatory attributes configured in NetX, a red exclamation point will indicate that these do not yet contain values.
      
      
   2. Like the primary NetX interface, mandatory attributes are displayed at the top and user attribute sets can be selected. Fill out the appropriate attribute fields. Entering values is completed much the same way as in the primary NetX application with dropdown menus, type ahead tagging, and date selection. Multi-select attributes have their values chosen by clicking the checkbox at the left. The Persist values checkbox allows NetX I/O to retain any attribute values saved so that multiple uploads can be completed without requiring that the attribute fields be filled in each time. If needed, the Reset button can be used to clear all entered values.
      
   3. After all the required values have been entered, click Save to return to the Upload tab. The red number on the Attributes button now indicates how many attributes contain values.
      
      
4. Once the desired attribute values have been entered and saved, the NetX folder chosen, and the desired local folders and files dragged to the upload gallery, click the Upload button. 
   
   This begins the upload process for the chosen files and file structure and takes you to the Reports dropdown in the Tasks tab.
   

The tasks tab

The tasks tab displays In Progress, Configure, and the Report as shown above.

In progress

This displays tasks currently being performed by NetX I/O as well as tasks that are cued to complete.

Reports

This dropdown displays detailed action reports on individual folders and files:

• The green up arrow indicates an action performed in your NetX instance. A down arrow shows a change in your local environment. A folder or file image will indicate which of those two is referred to in the line item.
• Possible Actions include Import (copying a file into NetX), Transfer (copying a folder into NetX), Delete (deleting a file from your local file directory), and Export (copying a file onto your local file directory). 
• Details indicates where the item may be found in NetX.
• Date gives the precise time of the action.

The Clear button will delete all line items in the Report currently stored in NetX I/O.

The Download button allows you to save a CSV of all the Report information.

The following columns are available in the CSV:

• Date — the date the event occurred, including timestamp
• Status — whether the action was successful (TRUE) or not (FALSE)
• Destination — where the action occurred; "client" means the local location, and "netx" refers to an event on the server.
• Type — possible types include: "warning" (message only), "asset" (an asset event), or "folder" (a folder event).
• ID — this is a non-zero value when: the Destination is "netx" and it corresponds to either the Asset or Folder ID. In the special case of a collision of a duplicate asset (same file name in the same folder), this will report as "-1".
• Path — this is generally the asset or folder path, but can also contain messaging.
• Action — possible actions include: "Warning" (message only), "Delete" (the asset or folder was deleted), "Transfer" (the asset or folder was transferred to the destination), "Import"  (the asset or folder was imported to the destination), "Move" (the asset or folder was transferred to the path), "Rename" (the asset was renamed to the path), "Lost" (the asset could not be imported, and was therefore moved to the Lost and Found).

Configure

This is where you configure the Modes used by NetX I/O. If no Modes have previously been configured, you are immediately taken to set one up when you open the Configure pulldown. 

If Modes have been previously configured, and a new one is required, select the New Task button:

When setting up various Modes you may not choose the Top Level folder. This is by design, you can sync a top level (first tier) folder, but not sync the Top Level itself. Watched Folder mode is the exception; when running in this mode, you can set the NetX folder to the Top Level.

Your NetX instance must be available during the Mode processes. If NetX is not available, NetX I/O will halt. NetX I/O is capable of handling a restart of NetX as long as the restart completes between the process cycles. For example, if you set the Schedule to 15 minutes, NetX can be restarted during those 15-minute pause windows and NetX I/O will happily re-authenticate. If it can't authenticate, it will halt.

 

Configuration glossary

Mode: This is the set of processes run by NetX I/O. A more in-depth description of each Mode can be found in the above section: All About Modes.

NetX Folder: This is the folder in your NetX instance that is affected, or contains subfolders and assets that will be affected, by the processes in the selected Mode.

Local Folder: This can be any directory on a network share or individual computer. It is the local directory selected to have the files and subfolders (but not the top level named folder) affected by the Mode.

Schedule: The processes of certain Modes (Watched Folder, Merge, and Sync) can be run at regular intervals: every five minutes, every fifteen minutes, every hour, or every day, or one time. 

Replacement Character: Occasionally files and folders are created using characters are that illegal for that purpose in NetX (a list can be found at Illegal Characters in Asset and Folder Names). NetX I/O will replace illegal characters with the one chosen and continue the processes indicated by the Mode or, if no Replacement Character is chosen, ignore the file or place it in Lost and Found. 

Advanced Options:

Sync Delete Limit: This allows you to choose how many files may be deleted from the local directory to match the assets that have been removed or deleted from the NetX folder involved in the Sync. After the number of files is reached, any other files that conflict with the assets in NetX or are otherwise indicated for deletion will be moved to the Lost and Found folder.

Blind: Checking this box will instruct NetX I/O to skip any NetX-vs-Local analysis and send all files regardless of whether they exist on the other side. Blind can increase the transfer speed but becomes less efficient if a user decides to routinely copy over redundant files into configured watched folders. The default for this setting is off.

Static Source Files: By default NetX I/O checks source files and folders before upload to be sure they are not in the process of being copied over. If this box is checked, NetX I/O will not perform this integrity check. Use caution when enabling this setting; by default this box is unchecked.

Checksum uploads: For file uploads, specifying checksum uploads means file checksums will be calculated at both locations to ensure absolute file integrity during transfers. This is optional.

 

Configuring the import mode

The Import Mode is fairly straightforward. 

1. Click the Browse button to select a folder from the NetX folder tree. Select the folder and then click the Select button.
   
2. Click the Choose button to select the local folder where the needed assets are located and click Open.
   
3. As this is a one-time event, the only other option to set (or not) is if you would like to import the files Blind. 
   

4. Clicking Run begins the Import and takes you to the Report section. Clicking Cancel will take you back to the initial Configure list.

   

Configuring export mode

The Export Mode moves in the reverse direction of Import.

1. Click the Browse button to select the folder from the NetX folder tree containing the assets you wish to Export. Select the folder and then click the Select button.
   
2. Click the Choose button to select the local folder where the exported assets are to be located on your local system and click Open.
   

3. Clicking Run begins the Export and takes you to the Report section. Clicking cancel will take you back to the initial Configure list.

   

Configuring a watched folder

A Watched Folder is very similar to an Import, but the files (not the folders) are deleted from the local directory. 

1. To choose the destination in NetX, click the Browse button to select a folder from the NetX folder tree. Select the folder and then click the Select button.
   
2. Click the Choose button to select the local file directory where the folders and assets are located or to be located and click Open.
   
3. Schedule the frequency at which you would like the selected file directory to upload.
   
4. Select the Replacement character to be used (or choose none) for files and folders with illegal characters in their names.
   
5. If you choose to use the Advanced options, you may set NetX I/O to assume Static source files or import the files Blind.
   

6. Clicking Run begins the initial import of local files and folders and deletion of the source files in the designated local folder and takes you to the Report section. Clicking cancel will take you back to the initial Configure list.

   

Configuring merge mode

Merge is a two-way operation and can be scheduled. 

1. Click the Browse button to select a folder from the NetX folder tree to be Merged with a local directory. Select the folder and then click the Select button.
   
2. Click the Choose button to select the local directory you would like Merged to the NetX folder and click Open.
   
3. Schedule the frequency at which you would like the selected folders to merge.
4. Select the Replacement character to be used (or choose none) for files and folders with illegal characters in their names.
   
5. If you choose to use the Advanced options, you may set the Merge to assume static source files. By default NetX I/O checks source files before upload to be sure they are not in the process of having changes saved. Checking the Static source files box overrides this process.

6.  After the desired options are selected, click Run. This will begin the initial Merge. Clicking Cancel will take you back to the Configuration list.
   

   If you have selected something other than Once for your Merge, the process will be timed to re-run at the designated interval. If, for any reason your user is logged out of your NetX instance, the process will run again at login and intervals will be timed from that "initial" run.

Configuring sync mode

Sync mode follows the same configuration process as Merge, with a single exception. In the Advanced options, you can select the number of files that may be deleted from your folder structure before files begin to be stored in the Lost and Found folder. The default is 100 files, but this may be set to 10, 100, 1,000 or 10,000 files. 

1. Click the Browse button to select a folder from the NetX folder tree to be Synced with a local directory. Select the folder and then click the Select button.
   
2. Click the Choose button to select the local directory you would like Synced to the NetX folder and click Open.
   
3. Schedule the frequency at which you would like the selected folders to Sync.
   
4. Select the Replacement character to be used (or choose none) for files and folders with illegal characters in their names.
   
5. If you choose to use the Advanced options, you may set the Sync to assume static source files.
   
6. The Sync Delete Limit allows you to choose how many files may be deleted from the local directory to match the assets that have been removed or deleted from the NetX folder involved in the Sync. After the number of files is reached, any other files that conflict with the assets in NetX or are otherwise indicated for deletion will be moved to the Lost and Found folder.
   

7.  After the desired options are selected, click Run. This will begin an initial Merge. After the one time Merge, all other scheduled runs of the configured Sync Mode will be compared to the State Map established in the Merge process. Clicking Cancel will take you back to the Configuration list.
   

   If, for any reason, your I/O user is logged out of your NetX intance, the process will run again from the beginning at login and intervals will  be timed from that "initial" run.

   
   Changes to the chosen local directory: Moving files around, renaming files and updating files locally are all supported. But, combining either a move or a rename with a modification (changing the file's internal bits) to a local file is not supported. In this case, the NetX Asset will be deleted; and the locally modified file will be reimported. This is not ideal as any NetX attributes will likely be lost. If you need to do both actions, please be careful to perform updates in a separate sync execution from any file renaming or moving. It's worth noting the in the reverse scenario, where you update a file and move it in NetX, the file will be refetched from the server regardless.
   Folder/ Asset limitation:  You can sync assets that live in multiple folders in NetX, however, you can't sync with a NetX folder that contains assets that are also contained in other folders that are synced at the same time. If you have an asset in NetX Folder A and NetX Folder B; you can sync with either Folder and it works. If you have Folder A, and it has subfolders B and C with an asset that lives in both B and C, you can't sync with Folder A. This is because the multiple paths for a single asset are not supported. In this situation, NetX will log an ERROR and exit.

 

Lost and found

Lost and found is not a mode, but a folder set up in your local directory so that NetX I/O can manage files and folders with illegal characters, and backup server/local conflicts (see above). When local storage is enabled, within the local storage path, a "lostandfound" folder is created. When scanning the source directory for new files to process, if it finds files or folders with "illegal" characters, NetX IO will move those files and folders into the lostandfound (all paths are relative and maintained). Or, if conflicts between the server and the local, local files and folders will be copied here as well. Basically, all problems end up here, if configured.

However, if you want NetX I/O to automatically repair folders and files that contain these illegal characters, you can see the Replacement Character option as noted in the configuration instructions. 

The help tab

The Help tab is the jumping off point for assistance with NetX I/O. It displays About information, including the exact version number and buttons to lead to the NetX Knowledge Base, your Lost and Found folder, and access to the NetX I/O Logs.

The Learn More button takes you to the primary NetX I/O page in the Knowledge Base using your default browser.

The Lost and Found button will open the Lost and Found folder containing any assets from the target local folder neither deleted nor uploaded when Modes are processed. 

The Reset button allows you to reset the application's configurations. Clicking the Reset button will launch this information modal:

There, you can choose to reset the following configurations:

• Tasks — this will delete any configured Tasks.
• Preferences — various preferences (attributes, last tab selection etc) will be deleted.
• Account — resetting the account configs will delete any URL, username and password.
• Lost and Found — resetting the lost and found will delete the entire lost and found folder and all the contents it stores (if any).
• Database — the app stores partial imports in a local database; resetting this will delete that database.
• Reset entire app — this will delete all configurations, the next time the app launches it will be as if it were installed for the first time.

The Logs button opens NetX I/O logs. If these have been set to Debug in the Account Tab, DEBUG items will appear alongside WARN and INFO entries. You can also enable DEBUG logging in the logs popup window. If needed you may download the logs using the Download button at the bottom of the Logs panel.

Known issues

• Producer-level users and above only — NetX I/O expects only Producer-level users or higher. While NetX will warn lower-level users if they login, there are certain UI elements that may continue to appear to work for such insufficiently privileged users, those functions will not function. 
• Very long timeouts — in some extreme cases, where very large file sizes are uploaded to storage systems that copy very slowly, a 10 minute HTTP timeout may not be sufficient. However, the UI does not allow for timeouts longer than 10 minutes. In such cases, please use the CLI version of NetX to set the timeout to an appropriately (very) long duration. 
• Attribute mismatch when moving between two different NetX servers — If you configure NetX I/O with one NetX server, and then later reconfigure NetX I/O with a different NetX server, it's possible that the Attribute number bug on the Import tab may display incorrect information. In this case, please simply reset the app and reconfigure NetX I/O with the new NetX server.
• Version 4 of NetX I/O requires NetX 9+ — if you attempt to configure NetX I/O version 4 with a NetX server that is running a version prior than 9, you will be presented with the following warning: 
  
  This will also show up in the logs as an ERROR entry: