EasyPopulate 4.0 Support Thread

groovypockets:

zc1.5.5a upgraded site hosted on a CentOS 6.7 server running php5.3.3
EP4 Config and import error screenshots attached
EasyPopulate-4.0-4.0.35.ZC acquired from github here
https://github.com/chaddro/EasyPopulate-4.0

exported complete products csv for template Full-EP2016Jun13-1301.csv
confirmed categories exist
confirmed syntax for category^subcategory
confirmed date syntax
not using SBA
created single product csv for import ClassesForEveryone.csv
getting errors for character limit of category and sub category on import
tried increasing character limit for these fields in MySQL as suggested in earlier posts from this thread. I still get error like op.
category name and sub category names character count does not exceed limits but I still get error and import fails
What am I missing?

Should this version of EP4 work with zc1.5.5a? If not is there a roadmap for when it will? And or is there a version of EP4 that is known to work with a specific version of ZC1.5x?

Thank you for your hard work on this mod. My client who owns this shop had been using 1.2.5.7 in zc1.3.9h for many years without issue.

Huh... Hard to believe it, but apparently there was an incomplete thought in the code when trying to add this type of check for both UTF-8 related databases/stores as well as for those not (BTW, highly suggest running the DB2UTF8 upgrade on your clients computer unless they have a specific reason to not use UTF8 and to update the configure.php files, the language file(s), and I think there is one other location that needs to be updated, though the plugin should identify those for you.)

So, in admin/easypopulate_4_import.php
line 1138 change from:

                if ((function_exists('mb_strlen') && mb_strlen($categories_names_array[$lang['id']][$category_index]) > $categories_name_max_len) || (!function_exists('mb_strlen') && strlen($categories_names_array[$lang['id']][$category_index]))) { 

To:

                if ((function_exists('mb_strlen') && mb_strlen($categories_names_array[$lang['id']][$category_index]) > $categories_name_max_len) || (!function_exists('mb_strlen') && strlen($categories_names_array[$lang['id']][$category_index]) > $categories_name_max_len)) { 

As to operation with ZC 1.5.5.a, it does, though again one of the things primarily expected in ZC 1.5.5.a is that all things database relate to UTF-8 or as applicable utf8...

Correction shown here.

And sorry, I may have gotten ahead of myself about the database not being in UTF-8... It would appear that really it is that one or more multibyte related functions are not present which is not necessarily indicative of UTF-8 support. Sorry about that.

I think you might be onto something actually with the UTF8 thing. The product descriptions are full of special character issues after I upgraded to 1.5.5a. The client does use MS Excel even though I've advised them not to. However, in the older version of EP we were able to work out a workflow that allowed them to use MS Excel.

Anyhow when creating the copy of the db I accepted the defaults for encoding and coalition. I'm not even sure what the system defaults are for either. I'm going to try an upgrade explicitly defining these before importing the db and executing the upgrade. I'll post back here when I get there with my findings. Good to know it's expected to work in 1.5.5a. Thanks for the quick response!

groovypockets:

I think you might be onto something actually with the UTF8 thing. The product descriptions are full of special character issues after I upgraded to 1.5.5a. The client does use MS Excel even though I've advised them not to. However, in the older version of EP we were able to work out a workflow that allowed them to use MS Excel.

Anyhow when creating the copy of the db I accepted the defaults for encoding and coalition. I'm not even sure what the system defaults are for either. I'm going to try an upgrade explicitly defining these before importing the db and executing the upgrade. I'll post back here when I get there with my findings. Good to know it's expected to work in 1.5.5a. Thanks for the quick response!
There are some filters included/incorporated here as well. There are some Internet instructions about how to "truly" export a csv file from Excel which involves several steps when I last looked at it. Otherwise from report by others it can be done from Excel. When I was doing some testing a long while back the biggest heartache was the date format and converting/setting it to the php format of YYYY-MM-DD (which BTW, is a great sequence to support sorting by date). Anyways, Excel tends to offer date/time in a more "convenient" format for example xx Mon YY or Mon xx, YY.

I do plan to add the ability to switch to tab delimitation from within the admin in the near future, but in the mean time you could modify the variable at the top of admin/easypopulate_4.php to use "\t" instead of ","

A lot of different options, just some things have been left in the background instead of adding all types of "features"

Using Easy Populate 4.0.28 - Beta 01-03-2015. I have over 25K items in my inventory. While attempting to download my complete inventory file I get the error message

[an error occurred while processing this directive]

Refresh the screen with same result. When I go back I find 2 files of different file sizes - 10885815 and 10370426. When looking at these files some of my newly added items with Easy Populate are missing. To get those I must use the filters when downloading. When I download those smaller files everything works ok.

I also cannot use the search to find these newly added items. The wesbsite search for customers works fine. If I want to make a change to the listing I must pick pages and go up or down to find the item.

keith z:

Using Easy Populate 4.0.28 - Beta 01-03-2015. I have over 25K items in my inventory. While attempting to download my complete inventory file I get the error message

[an error occurred while processing this directive]

Refresh the screen with same result. When I go back I find 2 files of different file sizes - 10885815 and 10370426. When looking at these files some of my newly added items with Easy Populate are missing. To get those I must use the filters when downloading. When I download those smaller files everything works ok.

I also cannot use the search to find these newly added items. The wesbsite search for customers works fine. If I want to make a change to the listing I must pick pages and go up or down to find the item.

What is the characterset for your database, in your includes/configure.php, in your includes/languages, etc?

EP4 uses UTF-8, using/having Latin-xxxx or other such character sets may cause the search issue identified.

The "error" is related to processing time. There is a setting in the configuration window for this as well I believe ZC has a similar setting (overrides or attempts to override the php.ini assignment). Because the category is smaller than the entire catalog this is why such "individual" downloads are successful. And yes, a "second" download tends to be created with a separate time stamp in the filename. The missing "newly" uploaded EP4 items from the download is because the new items have a higher products_id than the point at which the timeout occurs and therefore not included in the large single file download.

Another factor of consideration was the extent of content upload.

mc12345678:

What is the characterset for your database, in your includes/configure.php, in your includes/languages, etc?

EP4 uses UTF-8, using/having Latin-xxxx or other such character sets may cause the search issue identified.

The "error" is related to processing time. There is a setting in the configuration window for this as well I believe ZC has a similar setting (overrides or attempts to override the php.ini assignment). Because the category is smaller than the entire catalog this is why such "individual" downloads are successful. And yes, a "second" download tends to be created with a separate time stamp in the filename. The missing "newly" uploaded EP4 items from the download is because the new items have a higher products_id than the point at which the timeout occurs and therefore not included in the large single file download. From the configuration/Easy Populate 4 screen.

Another factor of consideration was the extent of content upload.

From the easy populate screen - Internal Character Encoding: ISO-8859-1 DB Collation: latin1. From the file includes/configure.php - define('DB_CHARSET', 'latin1'). From includes/languages, etc. I cannot help because you file not give a file name and I do not know where to look for etc.

I have changed the Script Execution Time to 100000 seconds. No change as the error screen still comes up when downloading the complete inventory file.

The other language file would be either:
Includes/languages/YOUR_LANGUAGE/YOUR_TEMPLATE/locale.php
Or
Includes/languages/YOUR_LANGUAGE/locale.php
Or English in place of YOUR_LANGUAGE in the above order of precedence.

Have you considered converting your database to utf-8 using the plugin db2utf8? Is this something your business target could support or is latin1 required to support operation?

mc12345678:

The other language file would be either:
Includes/languages/YOUR_LANGUAGE/YOUR_TEMPLATE/locale.php
Or
Includes/languages/YOUR_LANGUAGE/locale.php
Or English in place of YOUR_LANGUAGE in the above order of precedence.

Have you considered converting your database to utf-8 using the plugin db2utf8? Is this something your business target could support or is latin1 required to support operation?

Includes/languages/YOUR_LANGUAGE/YOUR_TEMPLATE/locale.php - not found
Includes/languages/YOUR_LANGUAGE/locale.php - not found

Have you considered converting your database to utf-8 using the plugin db2utf8? Is this something your business target could support or is latin1 required to support operation? - I do not know how to do that.

Here is another post https://www.zen-cart.com/showthread.php?178498-Product-Search-works-in-Store-NOT-in-Admin

This is were I first noticed the problem. Thank you!

Another place to try to set the timer for ZC is in configuration->My Store->Admin Set max_execution_time for processes (default is 60).

keith z:

Includes/languages/YOUR_LANGUAGE/YOUR_TEMPLATE/locale.php - not found
Includes/languages/YOUR_LANGUAGE/locale.php - not found

Have you considered converting your database to utf-8 using the plugin db2utf8? Is this something your business target could support or is latin1 required to support operation? - I do not know how to do that.

Here is another post https://www.zen-cart.com/showthread.php?178498-Product-Search-works-in-Store-NOT-in-Admin

This is were I first noticed the problem. Thank you!

In ZC forums, typically something like YOUR_LANGUAGE or YOUR_TEMPLATE refers to substitution of the content associated with the words. So if you operate your store in English only the the path would be like includes/languages/english/YOUR_TEMPLATE/locale.php or if a file doesn't exist in the directory name used for your template then the system would fall back to includes/languages/english/locale.php.

Yeah, I saw that thread which was what led me to ask about how the 1400+ items were added.

The plugin db2utf8 is not complex to use. It has instructions and there are no files overwritten. The database should be backed up before using it though.

The files listed above would need to be modified (instructions tell you which). The process is relatively painless and quick.

I changed it to 600 with no luck.

Thank you. I will give it a try.

keith z:

I changed it to 600 with no luck.

Well, while I haven't gone back to review the history of development and possible explanation on the topic, I believe one of the reasons the dropdown options were provided in the upper area was to address the export of such a large database considering there is an import method offering a similar breakup (split).

I have seen this behavior (having to export some form of sub-group) needed for other large databases some smaller in total number of product but with an additional language.

Where would this debug.txt file exist? I have searched everywhere with no luck.

I went to my live site and I cannot import I get the following. As you can see "Printers & Printer Parts" is not longer than 50 chars.

SKIPPED! - Model: 100323280 - Category name: "Printers & Printer Parts" exceeds max. length: 50
SKIPPED! - Model: 100334815 - Category name: "Printers & Printer Parts" exceeds max. length: 50
SKIPPED! - Model: 100361358 - Category name: "Printers & Printer Parts" exceeds max. length: 50

mc12345678:

Sorry I didn't see this earlier (last message I had received was the one before this because I had not visited the site between posting of the two recent messages).

Actually the error log that is discussed in that window is the debug.txt file that is now included in your list of files when accessing the admin panel for EP4...

This is most likely a result of data existing in the file that is not "escaped"... Be nice to see what is in that log to understand what was returned to cause this error so that it can be prevented in the future.

AZComp:

Where would this debug.txt file exist? I have searched everywhere with no luck.

I went to my live site and I cannot import I get the following. As you can see "Printers & Printer Parts" is not longer than 50 chars.

the debug log file is in your temp/ folder ( the EP4 folder ).

For me it is not and I do have debug turned on in settings.

AZComp:

For me it is not and I do have debug turned on in settings.

...but did you see a debug log being generated ?
When it's generated, there's also a download link in easypopulate_4 page ( the admin one ) .

mesnitu:

...but did you see a debug log being generated ?
When it's generated, there's also a download link in easypopulate_4 page ( the admin one ) .

I see no link in the easypopulate page and no file is generated in that folder.

AZComp:

Where would this debug.txt file exist? I have searched everywhere with no luck.

I went to my live site and I cannot import I get the following. As you can see "Printers & Printer Parts" is not longer than 50 chars.

For this issue see post 2524

Your host is not supporting part of the multi-byte functionality and when I expanded support to allow continued operation, I left off a comparison which forces that error even for something with a single character.