4.8.2 Batch Processing Multiple Files using Analysis Template
Origin can perform Batch Analysis utilizing an previously created Analysis Template by either importing multiple files, or loading existing datasets.
This tutorial focuses on the former case of batch analysis - importing multiple files with a common (ASCII) file structure, then repeatedly applying the analysis template for the linear curve fitting to the every imported data file.
What You Will Learn
This tutorial will show you how to:
- Make an analysis template which contains a Result Summary Sheet to collect the output of batch processing.
- Perform batch processing with multiple data files using an analysis template
In general, the procedure of batch processing is divided into two parts:
- In preparation, make an Analysis Template which is an empty framework of a workbook. This workbook template contains a Result Summary Sheet to collect the outputs of your analysis (e.g.,, linear curve fitting) during batch processing.
- Perform the Batch Processing using the prepared analysis template for multiple data files.
Preparing an Analysis Template containing a Result Summary Sheet to Collect Outputs of Batch Processing
Before performing the batch processing, we need to make an analysis template which contains a result worksheet (which may contain a graph in a worksheet cell as shown in this tutorial) to collect the outputs of batch processing inside the analysis template workbook.
Note1: Hereafter, you can find the location of <Origin EXE Folder> by choosing "Help: Open Folder: Program Folder" menu in Origin. Similarly for <User Files Folder>, choose "Help: Open Folder: User Files Folder" menu.
Note2: If you want to skip this preparation step to make the analysis template, you can use the ready-to-use analysis template file, <Origin EXE Folder>\Samples\Batch Processing\MySensorData.OGWU (or .OGW in Origin 2017 or before). Also, if you only want learn how to make the analysis template regardless the batch processing, refer to the tutorial, "Creating and Using Analysis Templates".
- Let us import a data file as a model. In a new workbook, choose "File: Import: Single ASCII" menu (or, click button), navigate to <Origin EXE folder>\Samples\Curve Fitting\Sensor01.dat file in the ASCII dialog box, and check Show Options Dialog checkbox. Click OK. This opens the impASC dialog to set import options In the appeared impASC dialog, turn OFF the "Rename Sheet with (Partial) Filename" check box (under the "Rename Sheets and Books" node). By clearing this option, the imported data goes always the same data sheet name (in our case "Data" sheet). Also, click the arrow button next to Dialog Theme, and select Save to <Sheet> in the flyout menus. By selecting this flyout, Origin saves your import settings to the worksheet, and Origin imports the files always in the same manner. Click OK, and the data file is imported to the worksheet. Double-click the Sheet1 sheet tab and rename to the more meaningful name, "Data".
- Now, let us perform the linear fit as a sample run. Highlight column B, and choose the Analysis: Fitting: Linear Fit: Open Dialog menu. The Linear Fit dialog box will be opened. In the dialog, set Recalculate to Auto (because we automatize the batch process without manual intervention), and click OK to perform a linear fit. In the workbook, go to the FitLinear1 analysis report sheet. (In the Reminder box, you can answer Yes.)
- Now, let us add a Result Summary Sheet to this workbook as a collection place of the output of batch processing. Go to the FitLinear1 sheet, and find the Summary table in it. This table contains the important analysis results such as the slope and the intercept, and it is a good place to start constructing the Result Summary Sheet. To do so, click the triangle button next to the table name, and choose Create Copy As New Sheet flyout. A new sheet named Summary is created from the table.
- Now, let us design and customize this new Summary sheet for our purpose. We let it contain the Pearson's r statistics, and the fitted curve graph. First, click the button TWICE to add two new columns (columns G and H). Let us add the the Pearson's r to the column G - First, go back to the FitLinear1 sheet, right-click on Pearson's r value cell in the Statistics table, select Copy flyout. Next, go to the Summary sheet, right-click on the first data cell of column G, and choose Paste Link to paste the link of the r value. For the caption of this statistics, go to the FitLinear1 sheet, in the Statistics table, click on the caption text Pearson's r, and right-click and choose Copy. Then, switch to Summary, right-click the cell in column G, Parameters row, and select Paste Link flyout to paste the parameter label.
- In the similar way, let's add the fitted plot graph to the column H. First, go back to the FitLinear1 sheet, right-click the fitting result graph under the Fitted Curves Plot branch, right-click on the graph to select Copy context menu. Next, switch to the Summary sheet, right-click on the first data cell of column H, select Paste Link flyout to paste the graph as a link. Give the long name of column H Fitted Curve.
- Finally, let us save this workbook as an Analysis Template for batch processing. Choose File:Save Workbook As Analysis Template menu, navigate to your <User Files Folder>, and enter Sensor Analysis for the analysis template name. Click Save. You can close Origin - because you have already save the Analysis template, it is not necessary to save this project unless you just want to keep it as a practice record.
Performing Batch Processing with Multiple Data Files
Once you have made an Analysis Template including a Result Summary Sheet, it is quite easy to perform a batch processing to get the summarized Output from many source data files.
- In a new project, choose "File: Batch Processing..." menu (or, click button). "Batch Processing" dialog appears.
- At the Batch Processing Mode, select the Load Analysis Template so that we can use the analysis template we already have made.
- At the "Analysis Template", select the Sensor Analysis.OGWU (or .OGW in older versions) file in <User Files Folder> which we have saved in the previous section. (You can click "..." button to locate your analysis template.) Similarly, at the Data Source, select Import From Files from the drop-down as we want to import multiple files.
- At the "File List", Click on the "..." button, and select ALL source files Sensor##.dat (## is the sequence number of the file) from the <Origin EXE Folder>\Samples\Curve Fitting folder (use SHIFT or CTRL key to select multiple files).
- At the Dataset Identifier, select File Name so that the file name of every imported file will appear in the Result Summery Sheet as the data source.
- At the Data Sheet drop-down, select Data to specify the receiving worksheet of import in the Result Summary Sheet.
Similarly, at the Result Sheet drop-down, select Summary to specify the output destination of the Result Summary Sheet.
- The "Delete Intermediate Workbook" check box may be either checked not to produce a large number of intermediate workbooks/worksheets, or unchecked to preserve all intermediate outputs in addition to the Result Summary Sheet. (Note: In Origin 2018, even if you check this check box, you don't have to be worried about loosing the intermediate reports because "Open Book(s)" button in the Result Summary sheet can regenerate individual full report later.)
- Click OK. All seven data files will be processed as indicated by the progress bar. A Summary workbook is generated, which contains summarized analysis results from the seven Result sheets in the analysis template. Put your mouse cursor on any graph cell, the fitted result graph will pop up to let you see the details. In Origin 2018, or later, if you want to extract full analysis reporst or their individual graphs, highlight the desired rows, and press the "Open Book(s)" button.