Time required for importing .zolca databases

pascal.lesage
Posts: 30
Joined: 19 Sep 2014 21:31

Time required for importing .zolca databases

Post by pascal.lesage » 23 Sep 2014 16:00

Hi Forum,

I have tried importing the ecoinvent database using the import openLCA database import function.

This is what I did:
-->created a new database
-->clicked "import", selected "other/database import", selected "from exported .zolca file", and navigated to the appropriate .zolca file for the database of interest.

* The ecoinvent v2.2 unit process database imports like a charm.
* The ecoinvent v2.2 LCI database was not finished after 3.5 hours, and I had to kill the application (cancel operation did not function). Most datasets seemed to be there, but I could not be sure (I don't know how to count datasets in a folder).
* The ecoinvent v3.1 cut-off LCI database was not finished after 12 hours, and I had to kill the application (cancel operation did not function). Again, most datasets seemed to be there, but I could not be sure.
* I have just launched the import of ecoinvent v3.1 cut-off UP and await results - 7 minutes and running.

Here is my system info:
OS: Windows 8
Processor Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2201 Mhz, 4 Core(s), 8 Logical Processor(s)
Installed Physical Memory (RAM) 8.00 GB
openLCA seems to consume about 20% of my CPU and 25% of my memory during import.

Has anyone else been able to import these databases?
How long did it take you?
Did you find a difference in import time between an import in an "empty" database (with only reference values) and a database that already contained process data?

Thank you!
Pascal

msrocka
Posts: 18
Joined: 29 Jan 2013 10:07

Re: Time required for importing .zolca databases

Post by msrocka » 24 Sep 2014 09:45

Hi Pascal,

It should take only seconds when you use the "Import database" option in the navigation menu to import a zolca-file as a new database (right-click on the database tree when no other database is activated, see attachment.

Merging databases from zolca-files indeed takes some time (as database updates in other LCA software systems too). If you want to merge two databases (e.g. ecoinvent LCI results and ecoinvent unit processes) it is faster when you first import the larger database (the LCI results) via the ‘Import database’ option in the navigation menu and then import the smaller database (the unit processes) into this new database.

Best regards,
Michael
Attachments
olca_import_database.png
olca_import_database.png (11.88 KiB) Viewed 41757 times

pascal.lesage
Posts: 30
Joined: 19 Sep 2014 21:31

Re: Time required for importing .zolca databases

Post by pascal.lesage » 25 Sep 2014 04:50

Thanks Michael for your reply.

Based on your answer, here is what I did:
0-Deleted all existing databases (I wanted to start with a "clean slate")
1-Created an “empty” database with reference data
2-Deactivated it
3-Right clicked and selected “import database”
4-Selected “ecoinvent3_1_cutoff.zolca”. Import took about 2 seconds.
5-Repeated 3-4 for “ecoinvent3_1_cutoff_lci.zolca”. Import took about 5 minutes.
6-Activated “ecoinvent3_1_cutoff_lci.zolca”
7-Moved all processes to a folder (child of process folder) called “LCI” to be able to distinguish these datasets from the UP datasets I wanted to import to have a merged database
8-Clicked "Import" and selected “existing database”, then “ecoinvent3_1_cutoff.zolca” from drop-down menu.
The process has been going on for 12 hours now. I will leave it running for another 8 hours - then I need to turn off my computer. I know that merges can take a long time, but do these times seem OK given the task and my system?

Has anyone else succeeded in merging the databases? How long did it take you?

Thanks again,
Pascal

aciroth
Posts: 750
Joined: 09 May 2010 23:28

Re: Time required for importing .zolca databases

Post by aciroth » 25 Sep 2014 08:56

Hi Pascal,

what Michael was trying to explain:

1. deactivate all databases
2. select 'import database' and import the LCI results first, a new database will be created with the name of the LCI results, in some seconds
3. then, with this database active, select 'import database' again, and import unit processes into it. This will take - depending on your computer - one hour or so

Hope this helps!
Andreas

pascal.lesage
Posts: 30
Joined: 19 Sep 2014 21:31

Re: Time required for importing .zolca databases

Post by pascal.lesage » 25 Sep 2014 22:36

Thanks Andreas.
Indeed, the procedure I tried did not yield any fruits: all the process categories from the “importing” database (ecoinvent3_1_cutoff) were copied into the “receiving” database (ecoinvent3_1_cutoff_lci), but no process data was copied.
So I tried the procedure you outlined (Figures are in the attached picture)
1) I started with no database (Fig.1)
2) I imported the ecoinvent3_1_cutoff_lci database using the “Import database” function (Fig.2a and Fig.2b). The database imported fine in just a few minutes.
3) I activated the imported database (Fig.3).
4) I moved all processes to a subfolder called “__LCI” to be able to distinguish these from UPs in the future (Fig.4a and Fig.4b)
5) I used the “same” import database function and selected the ecoinvent3_1_cutoff.zolca database (Fig.5a). This unfortunately imports in a separate database (Fig.5b).
6) I therefore used the “other” Import function (Fig.6a, Fig6b), as I did yesterday. Instead of selecting “Existing database” (which would be repeating something that had not worked), I selected “From exported zolca file” and then selected “ecoinvent3_1_cutoff.zolca” (Fig.7). It’s been running for more than 7h on a relatively “quick” computer (see specs above).

Am I doing something wrong?

Thanks!
Pascal
Attachments
AllFigs.JPG
AllFigs.JPG (122.88 KiB) Viewed 41721 times

aciroth
Posts: 750
Joined: 09 May 2010 23:28

Re: Time required for importing .zolca databases

Post by aciroth » 28 Sep 2014 17:01

Hi Pascal,
importing into the larger LCI database saves some time but the overall time requirement depends a lot on your harddisc. However, based on this discussion, we have looked into it and found that the import speed into an existing database can be largely increased, by a factor of at least ten, it was not really optimised before. This optimised import will be integrated in the new upcoming 1.4.1 release. If you cannot wait for this (we plan to release in a week or so) then you can contact me directly, thanks.
Best wishes,
Andreas

msrocka
Posts: 18
Joined: 29 Jan 2013 10:07

Re: Time required for importing .zolca databases

Post by msrocka » 14 Oct 2014 17:20

Here is the link to a new beta version (1.4.1 beta5) which includes these performance improvements: http://sourceforge.net/projects/openlca ... 4.1_beta5/.

Michael

pascal.lesage
Posts: 30
Joined: 19 Sep 2014 21:31

Re: Time required for importing .zolca databases

Post by pascal.lesage » 11 Nov 2014 21:21

Worked like a charm, thanks for the release.

yanushka
Posts: 5
Joined: 23 Jun 2017 10:47
Contact:

Re: Time required for importing .zolca databases

Post by yanushka » 23 Jun 2017 11:58

Hi all,

I apologize to bring this topic up again but I have similar issues with prolonged time for import of zolca databases (namely ecoinvent) in openLCA 1.6. I have already read the comments above and saw the videos on youtube and I found out that I should first import the system process database (lci) as it is larger and then I can import the unit process database which is smaller.

It happens for seconds on the youtube video (https://www.youtube.com/watch?v=o2mqPFvpdMs) but it takes a lot of time on my computer and for the lci database it doesn't work and I should forcefully close openLCA to cancel the process, just like you have discussed here. I tried importing the smaller database first and it takes around 15-20 minutes.

I am aware that the speed depend on the computer and here are the parameters of mine:

OS: Windows 7 (64-bit)
RAM: 8 GB
Processor: Intel(R) Core(RM) i7-4770 CPU @3.40 GHz 3.40 Ghz

I follow the steps as you have duscussed: I first create new database and it must be open in order to import the ecoinvent database. (You've recommended to deactivate all databases before importing but in my case it is impossible).

I would appreciate any help and advice.

Yana

aciroth
Posts: 750
Joined: 09 May 2010 23:28

Re: Time required for importing .zolca databases

Post by aciroth » 24 Jun 2017 07:58

Hi Yana,
sorry to hear that you have issues with the import, the speed depends also a lot on the speed of the database and of the free space available. We are however also providing a combined package on Nexus which already combines the ecoinvent unit and system processes, and LCIA methods, with the idea that users to not need to combine the databases on their own. Restoring this combined package in openLCA as database is much faster than the database import, since the database import needs to check and compare all items prior to import.
Hth,
Andreas

Post Reply

Who is online

Users browsing this forum: No registered users and 15 guests