Difference between revisions of "Exporting XML files from BPS"
| (10 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
When the XML data files are large (> 1000 records or abcd:Units) it is currently necessary to export multiple files from the BPS and run an xquery script to join them into a single ABCD file. | When the XML data files are large (> 1000 records or abcd:Units) it is currently necessary to export multiple files from the BPS and run an xquery script to join them into a single ABCD file. | ||
| − | The following steps describe how to do this: | + | ====The following steps describe how to do this:==== |
| − | 1. export the data from BPS using | + | 1. export the data from BPS using ABCD2 search. After the mapping is complete within the BPS then select the "QueryForms" on the top menu as shown in the screenshot below: |
| Line 13: | Line 13: | ||
| − | After selecting "QueryForms" click on "ABCD2 search" from the list below the textfield, see the screenshot below. After selecting the "ABCD2 search" the field is automatically filled with the Query tags for the ABCD search. Please remove the <filter> element from the text as shown in the screenshot. | + | After selecting "QueryForms" click on "ABCD2 search" from the list below the textfield, see the screenshot below. After selecting the "ABCD2 search" the field is automatically filled with the Query tags for the ABCD search. Please remove the <filter> element from the text as shown in the screenshot. Then set the limit to 999 in the following tag: <responseFormat start='0' limit='998'> (the default limit is set to 10). Click the "submit" button. An XML file is displayed in the browser. |
| Line 19: | Line 19: | ||
| − | 2. Save | + | 2. Save the file as {dataset name}_1.xml. |
| − | + | For the next file click on the "back" button on your browser to return to the query form and modify the tag to <responseFormat start='999' limit='998'> . This retrieves the next block of data records. Press the "submit" button again and then save the XML file as {dataset name}_2.xml ... | |
| − | |||
| − | |||
| − | 2 | + | 3. Continue to export XML files as in step 2, in each case modifying the start (1998, 2997, 3996 and so on). When saving the file naming it with a sequential suffix (_1, _2, _3 ...) until their are no further records. In each step you retrieve 999 records. |
| − | + | 4. Create a new project in rebind (see also the instructions [[Data_upload_to_rebind_framework#Creating_and_viewing_unpublished_projects|here]]) and name it with a unique name e.g. HBGnummeriert (dataset name) and then create a collection called ‘abcd’ by pressing 'New collection'. Click on 'abcd' to navigate into the new collection. Store all of the individual ABCD files in the 'abcd' collection by pressing 'Upload File' and selecting each file - one at a time until all files are uploaded: e.g. HBGnummeriert _1.XML, HBGnummeriert_2.XML etc. | |
| − | + | 5. Open another browser tab, enter the URL below. | |
| + | |||
| + | http://data-rebind.bgbm.org/rebind/join_abcd.xquery?coll=HBGnummeriert | ||
| + | |||
| + | ''Replace the final part of the URL with the name of your collection e.g. if your collection is called HBGnummeriert then the URL should end with coll=HBGnummeriert.'' | ||
| + | |||
| + | After pressing 'enter' the browser should show a white screen, once the script has run. | ||
| + | |||
| + | 6. Going back to the reBiND browser, click on your new project in the left hand panel and you see only one XML file (all of the individual XML files are now combined into one single), as in the screenshot below. | ||
| + | |||
| + | |||
| + | [[File:ReBIND_portal_project_overview.PNG|border]] | ||
Latest revision as of 15:50, 14 January 2015
Uploading large data files from BPS to reBiND
After the mapping of the data to the ABCD elements (http://wiki.bgbm.org/bps/index.php/ABCD2Mapping) the data needs to be either exported as XML files from BPS or uploaded directly to reBiND (see the following sections for details).
When the XML data files are large (> 1000 records or abcd:Units) it is currently necessary to export multiple files from the BPS and run an xquery script to join them into a single ABCD file.
The following steps describe how to do this:
1. export the data from BPS using ABCD2 search. After the mapping is complete within the BPS then select the "QueryForms" on the top menu as shown in the screenshot below:
After selecting "QueryForms" click on "ABCD2 search" from the list below the textfield, see the screenshot below. After selecting the "ABCD2 search" the field is automatically filled with the Query tags for the ABCD search. Please remove the <filter> element from the text as shown in the screenshot. Then set the limit to 999 in the following tag: <responseFormat start='0' limit='998'> (the default limit is set to 10). Click the "submit" button. An XML file is displayed in the browser.
2. Save the file as {dataset name}_1.xml.
For the next file click on the "back" button on your browser to return to the query form and modify the tag to <responseFormat start='999' limit='998'> . This retrieves the next block of data records. Press the "submit" button again and then save the XML file as {dataset name}_2.xml ...
3. Continue to export XML files as in step 2, in each case modifying the start (1998, 2997, 3996 and so on). When saving the file naming it with a sequential suffix (_1, _2, _3 ...) until their are no further records. In each step you retrieve 999 records.
4. Create a new project in rebind (see also the instructions here) and name it with a unique name e.g. HBGnummeriert (dataset name) and then create a collection called ‘abcd’ by pressing 'New collection'. Click on 'abcd' to navigate into the new collection. Store all of the individual ABCD files in the 'abcd' collection by pressing 'Upload File' and selecting each file - one at a time until all files are uploaded: e.g. HBGnummeriert _1.XML, HBGnummeriert_2.XML etc.
5. Open another browser tab, enter the URL below.
http://data-rebind.bgbm.org/rebind/join_abcd.xquery?coll=HBGnummeriert
Replace the final part of the URL with the name of your collection e.g. if your collection is called HBGnummeriert then the URL should end with coll=HBGnummeriert.
After pressing 'enter' the browser should show a white screen, once the script has run.
6. Going back to the reBiND browser, click on your new project in the left hand panel and you see only one XML file (all of the individual XML files are now combined into one single), as in the screenshot below.
