Hello,
My name is Rui Pimentel and I've more than 6 years of experience in development of web crawler applications.
I've read your whole project description and completely understand what you need.
I've already worked in a lot of web crawlers that worked as cron in webshops, to keep everything updated, such as price, stock and of course add new products. Some of them were magento, so I already have a good understand of how import products to there.
I'm ok with the backend, and I'm sure we'll think of one or other feature that could be useful to add there, and I'll do that with no problem.
The multi thread is also ok, it will make everything run smoother, and I've already done it in a lot of web crawlers before. About the proxies, they probably will make the process a bit slower, but it will depend on how fast the proxies are. If you don't want to waste any money on this, I would suggest to automaticly pull the proxies from [login to view URL], I've already used a lot of times and it's not that bad.
For last, the archive, seems something simple as well, probably we could just pull that products to a category that's not visible by the users, or even pull them to the backend, what you prefer is ok for me.
I hope I can work for you on this, you can see that I've a lot of feedbacks in works like this, and I assure you'll be happy with the final result.
I'll be waiting to hear from you and I'm looking forward to work for you on this.
Best Regards,
Rui Pimentel