wtonydanny 发表于 2017-7-30 01:02:20

這網站採集不到...

http://www.my-fresh.com/Store/Web/static-builder/product.html?id=c55fda51-0c0a-4ea0-9028-09d2453f11ab#filter=category&key=ca9094ec-1dbe-45a4-b5b8-24bb0f8aeca4&page=1

leweizxl 发表于 2017-7-31 08:47:20

可以获取需要用fiddler分析获取是post网址

longouxiu 发表于 2017-7-31 15:03:56

http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx抓包这个 数据里有 post的

utang112501 发表于 2017-8-17 10:29:59

用FD抓包得出真实网址：http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx这个POST网站浏览器时不识别的，要转化成GET---就是在这个网址后面加上？和FD抓取的数据包数据组合得出的网址
也就是http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx?where=Enable%3D%3Dtrue+%26%26+Visable%3D%3Dtrue+%26%26+Id%3D%3D%22c55fda51-0c0a-4ea0-9028-09d2453f11ab%22&orderby=Sort+DESC%2C+Number+DESC&select=new(Id%2CTitle%2CNumber%2CSerial%2CImage%2CImages%2CBrandId%2CVendorId%2CCartId%2CCategories%2CIntroduce%2CContent%2CSpecification%2CInspection%2CIsStockEnable%2CStock%2CIsVariantEnable%2CVariants%2CCreateOn%2CListPrice%2CPrice%2CIsOnSale%2CSaleStart%2CSaleEnd%2CSalePrice)&take=1&path=products

再把这个链接精简删除一下得出：http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx?where=Enable%3D%3Dtrue+%26%26+Visable%3D%3Dtrue+%26%26+Id%3D%3D%22c55fda51-0c0a-4ea0-9028-09d2453f11ab%22&path=products这个精简的页面应该就是你要采集的页面了。。。

utang112501 发表于 2017-8-17 14:50:32

用抓包工具分析得出POST网址是 http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx
要把这个POST转化为GET才能再浏览器看到。要在网址后面加上？再加上FD抓包得出的数据包组合一起-----
得出以下链接：
http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx?where=Enable%3D%3Dtrue+%26%26+Visable%3D%3Dtrue+%26%26+Id%3D%3D%22c55fda51-0c0a-4ea0-9028-09d2453f11ab%22&orderby=Sort+DESC%2C+Number+DESC&select=new(Id%2CTitle%2CNumber%2CSerial%2CImage%2CImages%2CBrandId%2CVendorId%2CCartId%2CCategories%2CIntroduce%2CContent%2CSpecification%2CInspection%2CIsStockEnable%2CStock%2CIsVariantEnable%2CVariants%2CCreateOn%2CListPrice%2CPrice%2CIsOnSale%2CSaleStart%2CSaleEnd%2CSalePrice)&take=1&path=products

再把链接精简---把URL中不必要的字符去掉（两个&字符间）
精简后链接是：
http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx?where=Enable%3D%3Dtrue+%26%26+Visable%3D%3Dtrue+%26%26+Id%3D%3D%22c55fda51-0c0a-4ea0-9028-09d2453f11ab%22&path=products
用URL解码器解码可以看出这个变化就是id="c55fda51-0c0a-4ea0-9028-09d2453f11ab"所以要采集其他产品可以用POST请求找出ID去采集就可以了。。。


longouxiu 发表于 2017-9-19 08:56:55

自己抓包后 查看 我给你 那个网址 就知道了
页: [1]
查看完整版本: 這網站採集不到...