HTML crawler cannot access products when shop needs login credentials

Support
#1

Hi,

I plan to have a shop that can only be accessed after the customer logged into my website (based on GRAV CMS).

This however prevents the HTML crawler to check the shopping cart upon the validation step, because obviously the crawler sees only a login site.

Any idea or concept how I can make the validation work for a shop that is behind a user login?

Thanks a lot for suggestions

#2

Hi @solinahrung,

You could have your “data-item-url” point to a page that is public that contains either the product definition as HTML or a JSON file.

If you don’t want to expose the product definitions on a public page, you can email us at geeks@snipcart.com, and we’ll provide you with a list of IPs that our crawler uses so you can whitelist them on your end.

Let me know if that helps,

Cheers!

#3

Thanks @Michael,
I’ll think about the options.

Whereas option 1 includes kind of two place with product definitions, double the maintance work, if not automatized. I’ll have to check whether I can export the product definitions automatically to a public URL.

Can you tell me if option 2 means, that I have to update the IP list every now and then? If so, how can I check that the IPs are still up to date? If the shop will fail validation, I will have lost some customers in the worst case…

But nonetheless thanks for the ideas.

#4

@solinahrung, we currently have a fixed set of servers using static IPs, so they don’t alternate frequently if ever. But you’re right, if we do decide to change our infrastructure down the road, it may cause the order validation process to fail.

The first proposed approach is definitively more optimal.