hi,
is it possbile to load a tsv file via ftp using this mod?

would the file have to be reformatted?
I've never used the mod before.

Hi, I found this quote below in another thread for my problem, but no word as to whether or not it was any help.

First my issue:

When trying to create an EP file in temp (it is already too big to just download) I got the following message:

Fatal error: Allowed memory size of 100663296 bytes exhausted (tried to allocate 96019431 bytes) in /home2/pccalca1/public_html/elocin/easypopulate.php on line 1041

A browse through the EP support thread led me to the quote below, but I was hoping someone else might be able to shed further light, as there was no response from the person that originally posted the issue.

Thank you!

awev:

Hello MikeyG,
You may of ran into a problem that you have no control over.

The above error message is generated by PHP. Translated it means that PHP is configured to use 8MB of RAM, and it tried to use about 3.6MB for EP. That means that other processes is using at least 4.4MB of RAM, so there is not enough RAM left that has be allocated to PHP to execute the command - in this case EP.

With this being an issue with PHP you may be restricted in what you can do, depending on if you host the site yourself or if it is being hosted. If you are hosting the site yourself you can just change the php.ini file, and increase the requested memory to 32MB, or anything else you may chose (just don't go overboard). If it is a hosted site you may not be able to change things yourself, just depends on what is in the agreement, and how comfortable you feel changing things. If your server is co-located then you should be able to change the file, otherwise you may have to speak with tech support and see if they are willing to change it for you.

A lot of things will just depend on what you are allowed, able, and willing, to do. If your hosting provider will not allow changes to the php.ini file then you have to find other solutions. If you are at this point then you have to make some hard choices. You will have to disable something, or just turn something off while trying to work with EP. You can't turn off ZC, yet you may have to turn off something like an image module. If this is not an option then see below for the final idea.

Worst case:

Worst comes to worst use the SQL patch tool under the tools section. You would start by installing MySQL (or another SQL RDBM [Relational Data Base Management] program of your choice) on one of your computers, maybe even doing a complete install of Apache, PHP, MySQL and Zen Cart with modules - for local testing. It may be a good idea to use the ZC install script to create the database so both the local and remote databases match in structure.

Once you have MySQL installed you can then use LOAD DATA INFILE to import your product information from the original source(s). Preform any needed operations to work the data into a form you need. Now export the database - you can use the backup command, and make sure it includes the needed commands, such as insert and update. You may have to edit the file in a text processor so as not to introduce unwanted control information from a word processor, and to insure the commands are correct.

Upload the file to your site, and then use the SQL patch tool to import it, or the MySQL tools included with your hosting package. You should be able to just execute things as all of the information and data is there. You could also brake the file up into small chunks to help fit within the memory constraints.

This this the hardest way of doing things as you have to know something about MySQL to help insure everything goes correctly. Also, you need to know how ZC lays out it's database, what tables you have to update, etc. Because of the technical nature of things this is not for the faint of heart.

Good luck Mikey G

tbrides - ftp your file to the /temp folder in Admin and then enter the file name in the : Import from Temp Dir (temp/) box in EP main page.

bifficus - Not an expert at PHP but it doesn't have a very big memory so I would suggest your file is too big,How many/ how big is your file/ qty of products? I would only up load max. 500 products at a time, maybe a few less for good measures.

Is there any interest for a file filter program to filter csv - tab etc (possibly future plans xml if any interest) files from Suppliers (csv/tab) catalogues or epos database exports to the Easypopulate Product import format, so no more cutting/pasting in excel or OO.

It is a python/glade application with the ability to pick whole or part field valves of the csv file and add additional valves globally.
As I see the only issue I have at present is the categories and the weight as these would need to be put in by hand, I am looking at possibly key words in the description to get around the categories issue.

Hi Scrat,

I am definitely interested in some supplier related features for zen-cart. What exists is few and far between. Ironically, I caused my problems in EP by adding some supplier related fields to the product (cost and supplier). My intention was to use this to easily generate price sheets for resellers.

With respect to my PHP issue, you are right, as was the other recommendation to increase memory. Guess how much I had to increase my PHP.ini memory to?

512mb... yes, 512mb. My database is tiny, but contains many catagories with linked products. A few of them also have rather lengthy descriptions. The net is that while my SQL database backup is 520kb, the EP file generated in the temp directory was over 500mb in size.

EP works really well, but it seems I have surpassed its capabilities.

I may still be able to use it, as you say, to add products bit by bit. I was already having to split files and add them that way. After a little trial and error I did get that working. Not sure now if it would work, due to the memory issues.

I am going to try and learn how to work with SQL directly to pull the reports out that I need like that. I know next to nothing about sql, but I think that would be the way to go.

Mack32:

Not sure if this is where I post this?

I have easy populate installed. When I try to download the Model/Attributes file to either file or to Temp. This is what happens.

I click on the link to download the file. It acts like it is doing something then goes to a blank page. No offer to save to file. nothing.

When I click on save to temp it acts like it does something and goes to a blank page. I check the temp folder and it has a file in there. Has most of the products listed.

I get no error messages of any kind.

I do have SEO urls installed. Not sure if this would change anything or not.

I am using Zen cart 1.3.7 and easy pop 1.2.5

any help would be appreciated.

Mack
I need help with this pleassee...it does it on all downloads if I try to get it to download to my computer or to temp dir?

Any help would be appreciated

Does anyone know how to move products into from an old category to a new category correctly? I did move them but the master category id is still the same as the old one whereas the new category becomes the linked category of the old category. This doesn't make sense because there is now nothing in the old category after moving. Please advise.

Mack32 - You really need to do some fault finding, I would remove EP and reinstall and see if that works, is your product database every big? Check your database for errors, partially delete products, it may be a hosting issue, has it ever worked properly if so what was the last thing you installed and remove that and see if that works!, again has it worked properly before contact your host to see if they have changed/upgraded anything php,mysql etc as this can affect things.
I have had a similar issue when I was with another hosting company since I have moved to Camelot Hosting things are so much better.

dealbyethan.com - I haven't tried this myself but if looks if it is leaving references in the database , I would delete the original record by using '9' in the status column and on the next row insert again the new record in the new category, under the usual status of '1'. Therefore it will completely delete the record and not leave any refs.

Thanks for your suggestion. If I don't want to delete the category but only to move them, do you have any other suggestion?

The main reason I asked to do this is because the old category is still the master category after its products are moved to the new category. So if I delete the old category, the new category will be deleted as well.

dealbyethan.com - Sorry I did not explain that clearly, you would not be deleting the whole catergory you are only deleting the product and then re-posting that product in the new catergory, this should not delete any catergories, just that product.

Yes, I get the idea. Some of my pages have ranking. I prefer not to lose them if possible.

Found the solution:

when you move a master product using EPA it should reset the master category id to the new category. IF not, please try the "Reset ALL Products Master Categories ID" button in Admin -> Tools -> Store Manager.

When I run Easy Populate, the results page never loads. It just spins until it either 1) displays a blank page or 2) asks me to open or save the file easypopulate.php (similar to if you have clicked a download file link). Everything works as in the products get inserted/updated as they should, but I like to see the summary page to easily check that the items were handled right (i.e. my data file was formatted correctly)

Anything I can do to fix this or anyone know what the problem may be. I'm not sure when this started or why. Not a huge deal if I can't resolve it I guess.

monkeytown:

When I run Easy Populate, the results page never loads. It just spins until it either 1) displays a blank page or 2) asks me to open or save the file easypopulate.php (similar to if you have clicked a download file link). Everything works as in the products get inserted/updated as they should, but I like to see the summary page to easily check that the items were handled right (i.e. my data file was formatted correctly)

Anything I can do to fix this or anyone know what the problem may be. I'm not sure when this started or why. Not a huge deal if I can't resolve it I guess.

I did a fresh install after a server crash corrupted my cart (it wasn't live yet fortunately) and I noticed this issue start when I had around 22000 items loaded in the cart. Now the results page never shows if I load 5k or 200 items from a .txt file. Any tips or an explaination appreciated.

monkeytown:

I did a fresh install after a server crash corrupted my cart (it wasn't live yet fortunately) and I noticed this issue start when I had around 22000 items loaded in the cart. Now the results page never shows if I load 5k or 200 items from a .txt file. Any tips or an explaination appreciated.

That sounds about right. Mine went into that mode after I passed the 20K products also. This has been mentioned a time or two in this forum but it doesn't look like there is a fix for it (at least no one has suggested one yet). I just reconciled myself to the fact that I don't need to know the details of my uploads, as long as they all get there in one piece. I tried a one line upload just to verify that file size wasn't an issue. It wasn't, as I timed out there, too.

Good luck in finding out anything more about this quirk!

Hi monkeytown - What Version of EP are you using? (Hopefully 1.2.5.4, if not use it or EPA)
Otherwise it could be a hosting issue if you are still having the issue on a new install as in theory you have done a clean install again. Contact you host and seek their advice if anything has changed on you account/server.

Wow - 22000 products I wouldn't like to do a stock-take, I would only upload 300-400 products at once as php does have a very big memory...

Also have you tried saving to \temp and ftp the file, as with bigger db's this can be a better option?
I was having similar problems with my old host EP would not update and hang, or take 'hours' to update or download. It may be a case that there are too many accounts on your server and its just too slow, I know I used to have only 10 connections to my database so if you have 9 users on your site as you upload you file that becomes 10 users and then think you could have 2000 other accounts on that server, well what do you think things begin to become very slooooow.
If this may be the case you may what to look for another host?

awhfy99 & monkeytown - Have you tried this on your test site on your localhost to see if you get the same results? If you don't it may be an issue that you need a dedicated server to get this to work with this many products.
Guessing, on a shared server, you will be limited to a shared amount of memory for each account?

monkeytown:

When I run Easy Populate, the results page never loads. It just spins until it either 1) displays a blank page or 2) asks me to open or save the file easypopulate.php (similar to if you have clicked a download file link). Everything works as in the products get inserted/updated as they should, but I like to see the summary page to easily check that the items were handled right (i.e. my data file was formatted correctly)

Anything I can do to fix this or anyone know what the problem may be. I'm not sure when this started or why. Not a huge deal if I can't resolve it I guess.

Monkeytown: May I ask where your hosting your site at and store? Very interested to see how your performance is with zencart and a store of that size.

For your issue: do you have access to php.ini? I do at bluehost.com and was able to up my memory size to 32meg and execution time. Most basic installs are only 8meg. Ask your hosting what these setting are and if you can get more. You can also check in phpMyAdmin under "variables" for you max user connections. Mine is 30 which is pretty generous for shared hosting.

During an update, you should definitely put the store on "maintenance" so no one is poking around while during upload.

For me, I can upload about 2,300 products and it takes about a minute to upload > process > output results.

Perhaps you can break the file up into 5k chunks?

I also wish the output was sent to a text file on the server.

Hope this was helpful.

chaddro.

Yes, I am using EP 1.2.5.4. and actually, the store has around 36K items total :/

We are currently hosting with a local company. I don't have access to the php.ini file, but they are pretty good about editing setting etc upon request but unless I knew specifically what to ask them to do, I think they would tire of me constantly asking them to tweak things to try and come up with a solution. We may in the future move to IPOWER and get on a Virtual Private Server.

I broke down my upload files into categories which are usually a couple hundred lines, although some are approx 1500. Then I have around 30K items that I broke into 5K files to upload in sections.

I am pretty sure I have the same problem on localhost, but I haven't loaded on it recently so I may not be remembering correctly.

Our store isn't really live yet we are building it. Technically it is, but no one knows about it so I haven't put it on maintenance mode for that reason. Regarding performance, it seems a bit slow in some places. In Admin, loading the top level categories takes probably 20 seconds, and while browsing, product detail pages take several seconds, and searching takes 10-15 seconds probably, but it doesn't seem too terribly long.

Under max user connections, on my phpmyadmin it says 0 :huh:

I agree that a text file would be nice at least in this case. I will maybe try to troubleshoot this sometime, but I guess for now it isn't a big deal.