× Looking for a useful answer?

Read this first and make sure you checked all steps: csvimproved.com/support/getting-started

Don't just say it doesn't work but describe in detail what is happening and add the following information:
* Version of the extension you are using
* CSVI version (Latest is not a version number)
* Sample of the file being imported
* A copy of the template you are using. Use the maintenance menu to create a backup of your template
* Collected debug information

Without this information you won't get a useful answer. If no debug log is attached, you will get a request to post it.

Where to get the debug information?

A tutorial ( csvimproved.com/support/questions-and-answers/390-how-to-collect-debug-information ) has been written to help you collect your debug information.

8 hours to import a 4mb file with 17 fields and 12500 lines

More
1 month 2 weeks ago #60401 by tharuna
tharuna replied the topic: 8 hours to import a 4mb file with 17 fields and 12500 lines
Hello,

So with that new information, is there a way to track the actual import time?

We need to check your latest debug log. You can increase Number of lines in log file from CSVI Dashboard >> Click Options >> Site tab and run the import again. Debug log will also tell us if the time has improved with the select query which was taking 3 seconds previously.

Any other information that you can provide to help with this process is greatly appreciated.

Can you also post your import file? We will run the import on our local server and check if we can reproduce your issue. You can send it as an email or in a private message.

Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory

Please Log in or Create an account to join the conversation.

  • cpointcc
  • Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
More
1 month 2 weeks ago #60402 by cpointcc
cpointcc replied the topic: 8 hours to import a 4mb file with 17 fields and 12500 lines
I have attached the log file. It is inconclusive. As far as sending the import file, the current has proprietary info in it. When I get the next import that will exclude that I will send it. It may not be today.

What email do I send it too? I don't see where to connect via private message.
Attachments:

Please Log in or Create an account to join the conversation.

More
1 month 2 weeks ago #60403 by rolandd
rolandd replied the topic: 8 hours to import a 4mb file with 17 fields and 12500 lines
Hello,

The logfile at least points out that the index on that SKU field didn't do anything because we are still waiting for 3 seconds to get an answer. What I do spot now in your screenshot is that your SKU index is 191 characters long. I am guessing you don't have SKUs that are 191 characters long ;) Reducing this number to a value that is more representative for your SKUs, perhaps something like 25 maybe, may help. Simply because there is less data to index and keep track off and you can get a faster hit.

It would be helpful if the server people can give any clue why this takes so long.

One other thing I noticed in the log is that you are resizing thumbnails, it may sometimes be useful to first run the product import and the image resizing in a separate import. This way you will have all the product info in place and the pictures come after that.

As far as sending the import file, the current has proprietary info in it.

Understood, no problem.

When I get the next import that will exclude that I will send it. It may not be today.

Thank you.

I don't see where to connect via private message.

You can click on our name to see our profile and there will be a button to send a private message. Alternatively you can also click on your own profile at the top right corner where there is a picture and it will have a link to the Private Messages area. To email, you can use This email address is being protected from spambots. You need JavaScript enabled to view it..

Kind regards,

RolandD

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory

Please Log in or Create an account to join the conversation.

  • cpointcc
  • Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
More
1 month 2 weeks ago - 1 month 2 weeks ago #60404 by cpointcc
cpointcc replied the topic: 8 hours to import a 4mb file with 17 fields and 12500 lines
Our server staff is asking if you have any specific optimization settings that are optimal for your program. The MySQL has not been optimized, but any input you have would be helpful. Its a fairly new server and tweaking it to fit this project is a new task. I did however optimize all the tables in the database this am.

No other clients have had issues with the server settings at this point until now with this project. If your product is completely dependent on the server optimization we need some criteria to start with.

I have changed the char count to 40 on the sku in the j2store_variants table.

The images: I am not importing images. We are only referencing the path in the import. They are already pre-sized. How or what can I change to avoid this?
Last edit: 1 month 2 weeks ago by cpointcc.

Please Log in or Create an account to join the conversation.

More
1 month 2 weeks ago #60405 by rolandd
rolandd replied the topic: 8 hours to import a 4mb file with 17 fields and 12500 lines
Hello,

If your product is completely dependent on the server optimization we need some criteria to start with.

We have no special requirements for the database server to run CSVI. When we send a request and get an answer 3 seconds later it might be because the database server doesn't have enough resources to process it faster. As I mentioned before, I am not a database admin, so I don't know how to fine-tune a database server.

No other clients have had issues with the server settings at this point until now with this project.

Are they all importing the same amount of data in the same setup using J2Store?

How or what can I change to avoid this?

In your template go to the Images tab and turn off the Create thumbnails, because that is now turned on. That should make a difference.

I was wondering, did you have the debug log enabled when you ran the full import? The debug log also takes time, so having that disabled when doing the full run will also make a difference.

Kind regards,

RolandD

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory

Please Log in or Create an account to join the conversation.

  • cpointcc
  • Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
More
1 month 2 weeks ago #60406 by cpointcc
cpointcc replied the topic: 8 hours to import a 4mb file with 17 fields and 12500 lines
OK we will continue to troubleshoot the server optimization.

Cannot compare client projects, just no issues in general. This project are only one using J2Store.

Images. there is no thumnail generating option.
Create image name set to no,
Image name based on
Image name format -JPG
Process images - No

Debug is not turned on. Only enable logging. I needed to see this for error troubleshooting.

One main issue I have just discovered. In my import I have a Product ID. That ID must stay the same. I don't see anywhere where it is being added. It looks like the product ID is being auto incremented. THIS IS A MUST. How can I insure the integrity of the Product ID to be based on the import file data?

Please Log in or Create an account to join the conversation.