Yoshimitsu Aoyagi
y-aoy****@hitac*****
2010年 2月 25日 (木) 21:10:07 JST
青柳＠日立ソフトです。 社内のWebサーバをクロールしているのですが、プロキシ経由でアクセスできず、 403が返って終了してしまいます。ちなみにウェブ認証の設定でプロキシサーバ を設定してますが、これはプロキシとは別なのでしょうか。 設定方法などご教授頂けるとありがたいです。 ＃ちなみに、web.xmlでorg.seasar.robot.client.http.CommonsHttpClient.proxyHost、proxyPort ＃を設定すると、fessサーバ自体がアクセス不能になります。 --- 以下、fess.out（*****でホスト名を隠してます） 2010-02-25 20:51:54,823 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Starting indexUpdater. 2010-02-25 20:51:55,160 [Robot-20100225205149-1-2] INFO org.seasar.robot.helper.impl.LogHelperImpl - Crawling URL:http://****/ 2010-02-25 20:51:55,353 [Robot-20100225205149-1-2] INFO org.seasar.robot.client.http.CommonsHttpClient - Checking URL:http://*****/robots.txt 2010-02-25 20:52:06,663 [Robot-20100225205149-1-2] INFO org.seasar.robot.helper.impl.LogHelperImpl - Crawling URL: http://go.microsoft.com/fwlink/?linkid=8180 2010-02-25 20:52:06,678 [Robot-20100225205149-1-2] INFO org.seasar.robot.client.http.CommonsHttpClient - Checking URL: http://go.microsoft.com/robots.txt 2010-02-25 20:52:06,696 [Robot-20100225205149-1-2] INFO org.seasar.robot.client.http.CommonsHttpClient - Could not process http://go.microsoft.com/robots.txt. go.microsoft.com 2010-02-25 20:52:06,697 [Robot-20100225205149-1-2] INFO org.seasar.robot.helper.impl.LogHelperImpl - Unknown host(go.microsoft.com): http://go.microsoft.com/fwlink/?linkid=8180 2010-02-25 20:52:58,811 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Processing documents in IndexUpdater queue. 2010-02-25 20:52:58,822 [IndexUpdater] INFO jp.sf.fess.solr.IndexUpdater - The number of a crawled document is 1. The processing size is 1. The execution time is 11ms. 2010-02-25 20:52:58,822 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://*****/ 2010-02-25 20:52:58,822 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Skipped. The response code is 403. 2010-02-25 20:52:58,833 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Updated 1 access results. The execution time is 11ms. 2010-02-25 20:52:58,833 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Getting documents in IndexUpdater queue. 2010-02-25 20:52:58,834 [IndexUpdater] INFO jp.sf.fess.solr.IndexUpdater - The number of a crawled document is 0. The processing size is 0. The execution time is 1ms. 2010-02-25 20:52:58,834 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Processed documents in IndexUpdater queue. 2010-02-25 20:54:02,247 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Processing documents in IndexUpdater queue. 2010-02-25 20:54:02,249 [IndexUpdater] INFO jp.sf.fess.solr.IndexUpdater - The number of a crawled document is 0. The processing size is 0. The execution time is 1ms. 2010-02-25 20:54:02,249 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Processed documents in IndexUpdater queue. 2010-02-25 20:54:08,460 [main] INFO jp.sf.fess.helper.WebIndexHelper - [EXEC TIME] crawling time: 133718ms
