大家帮忙看看这个网址怎么排除的
下面每组的两个网址的内容是相同的，但是网址不同，在采集网址的过程中能只保留一个吗？现在是有大量的这种网址在同一页面上。怎么排除啊？谢谢。
http://www.xxx.com/addiction-overview/methamphetamine-pills/pdfa--PDFA_dr_methamphetamine_pills.html;_ylt=Ar_q7eHgeubiI3kuHDZmhjqztMUF
http://www.xxx.com/addiction-overview/methamphetamine-pills/pdfa--PDFA_dr_methamphetamine_pills.html;_ylt=AkRQn9Bz3PjFrfSpGFtC646ztMUF
http://www.xxx.com/addiction-overview/why-our-ancestors-said-yes-to-drugs/pt--Psychology_Today_articles_pto-20020802-000010.html;_ylt=AgeywB8s9Ilb_qui8G9TSQ.ztMUF
http://www.xxx.com/addiction-overview/why-our-ancestors-said-yes-to-drugs/pt--Psychology_Today_articles_pto-20020802-000010.html;_ylt=AvYQhzkX9e_Fgf.RpYUkyxyztMUF
http://www.xxx.com/addiction-overview/addiction-a-whole-new-view/pt--Psychology_Today_articles_pto-19940901-000020.html;_ylt=AobL7ZnM4P.Et18l007jJ.6ztMUF
http://www.xxx.com/addiction-overview/addiction-a-whole-new-view/pt--Psychology_Today_articles_pto-19940901-000020.html;_ylt=At0VTewguODt44v9wxDQVZ6ztMUF
http://www.xxx.com/addiction-overview/how-to-stop-drinking-alcohol/healthwise--tj2649.html;_ylt=AorpLbelwK5f2iFzEWw5TWiztMUF
http://www.xxx.com/addiction-overview/how-to-stop-drinking-alcohol/healthwise--tj2649.html;_ylt=AnEv_H3rKA._vw7WsI2TDG2ztMUF
[ 本帖最后由 afteralls 于 2008-3-10 21:41 编辑 ]
页:
[1]