6.5.1 Connecting to File
Data Connectors build connections between an Origin worksheet/matrix and a data file on your PC or local network. Many common file types are supported.
For a summary of key differences between Connectors and older import methods, see FAQ-1020 Is there a difference between Connect and Import?.
Connectors for additional file types can be added by choosing the Add New menu item at the bottom of the Data: Connect to File menu. This opens the App Center to show only uninstalled Connectors. Click the Download and Install icon to install the Connector App on your PC.
Alternately, go to the Origin File Exchange on the OriginLab website and select a Connector from the list. On the App page, click the Download File button and when the download completes, drag the file to the Origin workspace. For App documentation, see the App page's Description section.
Adding a Connector to the Book
To add a Data Connector to the book and import data:
- With a worksheet/matrix active, choose Data: Connect to File.
- Select the desired data file from the Open dialog box.
This opens a Connector-specific dialog box where you can configure your import options.
- For multi-page files such as MS Excel or MATLAB, you will choose an initial sheet to import (e.g. Excel Sheet). Upon import, a collapsible Data Navigator panel is added to the book giving you access to the files remaining sheets.
- The name of the connected file is added as the book Long Name (does not apply to CSV or Origin files).
- When connecting matrixbooks to files, Image Mode and Image Thumbnails/Slider are automatically enabled when the number of imported matrices exceeds the value of system variable @MSS (default = 10). Does not apply to Origin files. For information on changing the value of a system variable, see this FAQ.
Use the Text/CSV Connector to import text files such as CSV, TXT, and ASC.
- Auto determine Main header lines or clear the check box and specify the main header and the sub-header, including column Long Name, Units and Comments, manually. Hint: a blue hint will show the number of auto-detected Main header lines.
- When main header lines are detected or manually specified, the import dialog shows a Save file header to drop-down list with options to save the file header to the Worksheet label area or the Book organizer (see image below).
- You can opt to Import first 50 lines as preview or for partial import.
| CSV Import Options
|| CSV Import Options, Main Header Lines
You can sort the whole worksheet by a selected column (right-click and select Sort Worksheet) after importing data by CSV Data Connector.
Data Block Markers
If your file is a mix of data and non-data, you can define those characters which delineate data blocks:
- Use the Start box to define the sequence of characters which mark the start of your data. If that sequence of characters uses section notation (i.e. is enclosed in square brackets as in [Data]), then End should be left blank.
- When using section notation, make sure Main header lines = Auto.
- Support for asterisk ("*") wildcard (e.g. "-*" will look for dash string of any length).
- When a blank line marks the end of the data block, you can specify End = <blank>.
- For Excel files which have multiple sheets, specify an Excel Sheet to connect (you will have the chance to connect additional sheets after the connection is made.
- If Main header lines are detected or specified, an additional Save file header to drop-down is added to the bottom of the dialog, with options to save the file header to the Worksheet label area or the Book organizer.
- To manually specify the location of Column Labels, check the box and enter rows containing Long Names, Unit, Comments From and Comments To.
- If your file contains multiple sheets, use the Data Navigator to connect to additional sheets, as needed.
Like Text/CSV data, data that you import using the Excel Connector can be sorted after import. Select a column, then right-click and choose Sort Worksheet.
The Data Connector Browser dialog will open.
The upper panel loads the structure of the imported file.
- Click on one branch to view the meta data in the right panel.
- Choose the desired branch and click the down arrow to add it to the lower panel.
- Click OK button to connect and import selected data.
- If your file contains multiple arrays, variables, etc., use the Data Navigator to connect to additional data, as needed.
|Note: When connecting to MATLAB, JSON and NetCDF files -- either local files or web files -- custom sheet names will be kept when reconnecting to your data file (Import or Import All Books from the Data Connector menu ). This applies only to file types that use a tree framework and does NOT apply to other file types such as Excel or Text/CSV.
The HDF Connector can be used to import HDF5 (.h5, .he5, .hdf5) or MATLAB's version 7.3 mat files stored locally or on the web. For more information, see this OriginLab File Exchange page.
Note that when you initiate a connection to a file, the Data Connector Browser opens, listing all variables in the file as a tree. There is a column on the right of each variable to indicate where each variable can be imported. Options include Worksheet, Matrix sheet or both, as shown in the following diagram.
You can connect directly to any node in the file. If a worksheet is active and the variable is marked with Matrix, you will be prompted to import the node to a matrixbook. However, you can opt to import the matrix data into a workbook and Origin will automatically add the node as a matrixsheet in the active workbook.
If you connect to the root of the file, the Data Navigator opens, allowing you to selectively connect to individual nodes. As mentioned, if the data are compatible with the active sheet, they are imported into the sheet. If another sheet type is needed, it is added to the active book.
Origin Connector imports data from an Origin project file. This allows you to use Origin project as a database to organize and store your data. From Origin 2020, the Origin Connector supports SQL query language for searching and extracting desired data.
- In the Select... dialog, click Select Query button to open Select Query dialog.
- Enter query strings in the edit box to, for example, search in the entire project and pull all the columns with the specified LongName and plot designation.
- Clicking OK button will add the query to the lower panel. Click OK again to import columns that meets the condition to a new workbook.
- In addition to query capabilities, you can use the Data Navigator to connect to additional sheets in the Origin project, as needed.
The Data Connector Browser dialog will open.
The upper panel loads all the tables on the web page. Choose the desired table and click the down arrow to add it to the lower panel. Click OK button to import.
Connect Multiple Files
Use this menu command to connect to multiple files of identical file type and structure.
- With the workbook or matrixbook active, choose Data: Connect Multiple Files.
- Set the Connector type using the Data Connector drop-down list.
- Set Source to Specified Files (pick individual files from a list) or Files in Specified Folder.
- Specify File Names or Folder.
- Check boxes to import files into the Same Book (worksheet only) or to Open Dialog for First File (useful for specifying how to handle file metadata).
If choosing Data Connector = Import Filter, see the next section.
Import a File with Filter
This option lets you use Data Connectors with an import filter file (*.OIF) that you created with the Import Wizard.
When Connecting to a Single File:
- With a worksheet/matrix active, choose Data: Connect to File: Import Filter.
- Select a desired data file from the opened browser.
- In the Select Filter dialog, select the desired filter. Click OK to import.
When Connecting Multiple Files:
- Choose Data: Connect Multiple Files or click the Connect Multiple Files button on the Import toolbar.
- Set Data Connector = Import Filter.
- Choose files and decide whether you want them imported into the same book or separate books, then click OK.
When choosing this option, there is no Open Dialog.... check box in the files2dc dialog box. Instead, an Import Filter dialog opens where you can browse to and select your import filter file.
This is another way to connect to multiple files. However, this is much more than a simple multi-file import mechanism. The real value of "cloning" is in rapidly duplicating graphing and analysis operations carried out on a prototype file, on subsequent files of the same type and structure. Once you have a prototype, cloning is trivial:
- Import your prototype data file using Data: Connect to File, then perform the desired graphing and/or analysis operations.
- With the data-containing workbook or matrixbook active, choose Data: Clone Import; or click the Clone Import button on the Import toolbar.
- Choose files and click OK.
See the tutorial "Importing Multiple Files into Workbooks Cloned from Active Workbook" for more information.
Connect Files by Appending Rows to Current Sheet
This method of connecting to multiple files is only supported by Text/CSV as of version 2021. Files must be appended one-by-one and will be imported by appending rows to the current worksheet.
- Use the Text/CSV Connector to import a file into the worksheet. Be sure to specify a Data Block Marker if your file uses one.
- With the worksheet active, open the Script Window (Window: Script Window), type the following and press Enter:
- Return to the active worksheet and click the green Data Connector icon .
- Choose Data Source, set File Path to the file you want to append and click OK.
- Choose connection options including Data Block Marker, then click OK to import the file by appending data to the current worksheet as new rows.
Other types added by Add New Menu
Add New menu will open the App Center dialog with all published Data Connector Apps listed. Choose the desired Apps to add to the Data menu.
Click on the App title to open a web browser to File Exchange page to see the detailed documents of that App.
Manually open the File Exchange field on OriginLab website. On the Apply Filter panel on the left, check Data Connector checkbox under Refine by Category. This will filter all Data Connector Apps. Click on the desired App to download that App and see the detailed documents on its own page.