EasyPopulate 4.0 Support Thread

What is the filename of the exported file and the then imported file? What was the method of "testing" the changes?

The name of the file exported was: Full-EP2016Apr27-145307 and we tried an import using the same name as we uploaded and as Full-EP2016Apr27-145307-import

Our attempt to test was simply to attempt a successful upload. No other testing method implemented.

Sorry... our attempt to test was to attempt a successful IMPORT, not upload.

Carbonless:

Sorry... our attempt to test test was to attempt a successful IMPORT, not upload.

Glad you provided that correction, because I was just about to comment on the difference between the upload and import (other versions don't seem to have that "two" step process or it is not understood the difference.

Glad to know that part, but how do you "know" that no changes were made to anything after the import was what I was trying to get at..

As to the "setup", well I consider this new feature potentially hazardous to the "unknowing" and so I did include a little bit of "difficulty" to set it up at least as I have just taken a lap through the code... That being, to export with the products_id included one needs to currently add it to the user defined fields list. For import, the v_products_id field must be present and the admin configuration switch should be set to something other v_product_model... Capitalization does count, so above where V_products_ID was typed, that would be an incorrect field name for the purposes of the code.

Also were the "number" of columns modified between the export and import? (ie. columns removed?)

My steps:
Import/Export Primary Key: products_id
Added:

Exported file
Changed ONLY the Product name of 2 items and modified the model names of the same 2 products.
Saved file
uploaded
clicked import

Result:

Finished Processing Import File

Updated records: 0
New Imported records: 0
Errors Detected: 0
Warnings Detected: 0
Memory Usage: 21087176
Memory Peak: 23515400
Execution Time: 0 seconds.

Verified against the actual products online with no changes.

I did verify that "For import, the v_products_id field must be present and the admin configuration switch should be set to something other v_product_model"

Any other item I'm missing?

I just ran an export with user defined field products_id in the admin-configuration (although was using version 4.0.34 not 4.0.34.a), then added three characters to the end of one of my test products, then with the primary key set to products_id, imported the file, every product showed as "updated" (even though technically nothing was to change except for the three characters added to the product name), and when I went to the store, the three characters were added to the product's name as expected...

Let me try modifying the product_model information of my test product to see what happens and if that is somehow associated... It was my expectation that this process would allow modification of the products_model field because that field would no longer be a primary field of concern...

Thank you for your time and feedback. May have to do a reinstall. I'll await your test result.

Okay, so it does appear that the products_model record for some reason is not getting updated, but other related information does... Need to chase that down because someone else that had requested the capability to import by products_id didn't mention this "issue" otherwise it wouldn't be one...

Here are the results of me importing the test products that were exported from my test category (single category only rather than the entire file) where the only change between export and import was the products_model for "model" 277 was to add three characters.:

Filename: Full-EP2016Apr27-192924.csv

UPDATED! - Model: 275 | prod_attr | 1 | | Test Multi | Trying to | | | | | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | | 2016-03-18 | 10000 | | Test attri | --none-- | 1 | 0 | 0 | 0 | 0 | 0 | | | |
UPDATED! - Model: 276 | test_singl | 1 | | Test Attri | This produ | | | | | 15 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | 2016-03-28 | 100 | | Test attri | --none-- | 1 | 0 | 0 | 0 | 0 | 0 | | | |
UPDATED! - Model: 277 | test_singl | 1 | Rdog1.jpg | Test Attri | This produ | | | | | 15 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | 2016-04-01 | 100 | | Test attri | --none-- | 1 | 0 | 0 | 0 | 0 | 0 | | | |
UPDATED! - Model: 278 | test_doubl | 1 | | Test Attri | This produ | | | | | 16 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | 2016-04-06 | 99 | | Test attri | --none-- | 1 | 0 | 0 | 0 | 0 | 0 | | | |
UPDATED! - Model: 279 | test_tripl | 1 | | Test Attri | This produ | | | | | 17 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | 2016-04-06 | 100 | | Test attri | --none-- | 1 | 0 | 0 | 0 | 0 | 0 | | | |
UPDATED! - Model: 280 | add_attrib | 1 | | test add a | This is he | | | | | 25 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | 2016-04-13 | 5 | | Test attri | --none-- | 1 | 0 | 0 | 0 | 0 | 0 | | | |

Finished Processing Import File

Updated records: 6
New Imported records: 0
Errors Detected: 0
Warnings Detected: 0
Memory Usage: 785064
Memory Peak: 1302016
Execution Time: 0 seconds.

What is being used to modify the file/store it as a CSV?

To confirm, you have the primary key set as products_id and have it entered in the User defined products field?

here is my configuration
Title Value Action Uploads Directory EPtemp/
Uploads Directory Admin/Catalog true Info
Import/Export Primary Key products_id Info
Upload File Date Format m-d-y Info
Default Raw Time 09:00:00 Info
Upload/Download Prices Include Tax false Info
Verbose Feedback true Info
Show all EP4 Filetypes with Files true Info
Replace Blank Image false Info
Make Zero Qty Products Inactive false Info
Smart Tags Replacement of Newlines true Info
Advanced Smart Tags false Info
Debug Logging true Info
Maximum Quantity Discounts 3 Info
Split On Number of Records 2000 Info
Script Execution Time 60 Info
Convert Curly Quotes, etc. 0 Info
Convert Character 0x92 1 Info
Enable Products Meta Data 1 Info
Enable Products Music Data 0 Info
User Defined Products Fields products_id Info
Export URI with Prod and or Cat 0

I downloaded a new file. I edited one product title by 5 characters.
removed, other than header line, all other unedited lines, saved the file with one record in it, uploaded and imported
the result was the same

Import Results

Filename: Full-EP2016Apr27-184246.csv

Finished Processing Import File

Updated records: 0
New Imported records: 0
Errors Detected: 0
Warnings Detected: 0
Memory Usage: 4867608
Memory Peak: 5363688
Execution Time: 0 seconds.

Huh.. Go figure.. Can add a new product with a new model number, but it looks like the products_model field for "updating" didn't get incorporated in the update section... (looking strictly at the code) even though I have it (products_model) included as a user defined field.

So, in this case? (until I get it fixed as part of my very soon to be released 4.0.35 as I want to test the few changes I made), one would need to or could, duplicate the product information in the import file, while importing by products_id such that the first occurrence deletes the product (status 9), the second occurrence with the same products_id reloads it... But, shucks I guess there is a downside to that as if there are attributes and other data "dependent" on the products_id, then they may get removed/reshuffled as well... Okay, should have a fix momentarily for the UPDATE of the products_model when using products_id or blank_new as the import option.. You should still be getting a report of updated product though or some sort of error that either the primary key doesn't exist or there is a problem with the remaining data...

Thank you for pointing out this issue... It's a shame, I guess the update portion hadn't been fully flushed out because the insert works fine... Maybe the priorities on the project changed, I can't recall..

Carbonless:

To confirm, you have the primary key set as products_id and have it entered in the User defined products field?

here is my configuration
Title Value Action Uploads Directory EPtemp/
Uploads Directory Admin/Catalog true Info
Import/Export Primary Key products_id Info
Upload File Date Format m-d-y Info
Default Raw Time 09:00:00 Info
Upload/Download Prices Include Tax false Info
Verbose Feedback true Info
Show all EP4 Filetypes with Files true Info
Replace Blank Image false Info
Make Zero Qty Products Inactive false Info
Smart Tags Replacement of Newlines true Info
Advanced Smart Tags false Info
Debug Logging true Info
Maximum Quantity Discounts 3 Info
Split On Number of Records 2000 Info
Script Execution Time 60 Info
Convert Curly Quotes, etc. 0 Info
Convert Character 0x92 1 Info
Enable Products Meta Data 1 Info
Enable Products Music Data 0 Info
User Defined Products Fields products_id Info
Export URI with Prod and or Cat 0

Would say similar:

Uploads Directory secret_folder_with_no_end_slash
Uploads Directory Admin/Catalog true Info
Import/Export Primary Key products_id Info
Upload File Date Format m-d-y Info
Default Raw Time 09:00:00 Info
Upload/Download Prices Include Tax false Info
Verbose Feedback true Info
Show all EP4 Filetypes with Files true Info
Replace Blank Image false Info
Make Zero Qty Products Inactive false Info
Smart Tags Replacement of Newlines true Info
Advanced Smart Tags false Info
Debug Logging true Info
Maximum Quantity Discounts 3 Info
Split On Number of Records 2000 Info
Script Execution Time 60 Info
Convert Curly Quotes, etc. 0 Info
Convert Character 0x92 1 Info
Enable Products Meta Data 1 Info
Enable Products Music Data 0 Info
User Defined Products Fields products_id Info
Export URI with Prod and or Cat 0 Info

Believe there is still an open question. What is being used to edit the csv file/save it?

If you in fact open the file with a plain text editor does it look correct? (typically quotes around each set of text, that is then separated by a comma, with some form of carriage return at the end of each line.)

Thank you. I will test first thing in the morning.... Using OpenOffice as per suggested!
Thank you for your well used time.

FWIW, I also changed the stored folder name for files to include a / at the end of it, reperformed import with same/similar results as previously reported...

Found and fixed the issue(s) with updating while not using products_model as the primary key. Haven't uploaded the fix quite yet, but will be up soon. Want to also check some of the changes made to the featured products area to address some previously pointed out issues.

Not yet sure about the 0 products updated issue... please check for any log errors in the logs directory...

Great news. I eagerly await your upgrade. Thank you for the time and attention to my request and giving great support.

Carbonless:

Great news. I eagerly await your upgrade. Thank you for the time and attention to my request and giving great support.

While I did notice a minor difference in the provided code and what was on my test site, I hope that it is the smoking gun for why import with product_id as the primary key was an issue. I also was trying to think back on why the problem existed here versus previous testing and development and it appears that in the spanse of local git work that the corrections may have existed on another branch or were incorporated specifically for the requestor and didn't make its way into a specific branch. :/ If nothing else, a reason to identify an issue when discovered rather than walking away. :) Thank you for identifying the issue and working with me to resolve. Of course proof to be seen.

I would have applied the changes to my 4.0.35 path, but I was going to apply it to the changes to improve the featured import. When I went to test the modifications to the featured import at least on ZC 1.5.5, I noticed that featured product control treats dates of 0001-01-01 as a date in 2036, so I need to look into that operation from the ZC side. But, once I get back to my computer I will apply the change of this discussion directly to the master for the branch and deal with the featured issues separately.

I have been with ZC since 2002 and appreciate the efforts of those that diligently support the community. I spent 2 hours reading the entire support thread yesterday before I even asked my question to find a solution. For those that have an established store with products that have been indexed by search engines and have back links, it would be devastating to simply replace the existing products. In my case it is imperative that the product id number stay the same to support backlinks.

Thank you for the great support and taking time from paid projects to bring a valuable service.

I realized you asked about how the file was opening.... here is the text version of the file with the first product
v_products_model,v_products_type,v_products_image,v_products_name_1,v_products_description_1,v_products_url_1,v_specials_price,v_specials_date_avail,v_specials_expires_date,v_products_price,v_products_id,v_products_weight,v_product_is_call,v_products_sort_order,v_products_quantity_order_min,v_products_quantity_order_units,v_products_priced_by_attribute,v_product_is_always_free_shipping,v_date_avail,v_date_added,v_products_quantity,v_manufacturers_name,v_categories_name_1,v_tax_class_title,v_status,v_metatags_products_name_status,v_metatags_title_status,v_metatags_model_status,v_metatags_price_status,v_metatags_title_tagline_status,v_metatags_title_1,v_metatags_keywords_1,v_metatags_description_1
MG SIGNS,1,buttons/magneticvehiclesigns.jpg,Magnetic Signs,"<table width=""600"" bgcolor=""#e1f0ff"" align=""center""> <tbody><tr> <td valign=""top"" bordercolor=""#333333""><div> <div> <div align=""center""><strong>Magnetic Vehicle Signs</strong><br /> Printed FULL COLOR & full bleed directly on a 30mil low shine magnetic material.</div> </div> </div></td> </tr> </tbody></table>",,,,,0,12,0,0,0,1,1,1,0,,5/2/04 3:30,0,,Magnetics^Magnetic Signs,Taxable Goods,0,0,0,0,0,0,,,