HttpClient weird behaviour

lars_kirchhoff Joined: 05/27/08 Posts: 59

Posted: 08/12/09 15:32:36 I'm using the HttpClient? to fetch a lot of websites from different domains (approx 130'000 in the test case). The problem happens when using HttpClient? in combination with threads (approx. 50). Between 10'000 and 20'000 requests I get a "unable to resolve <domain name>" for the rest of the domains. I tried to figure out where the error comes from. So far I could track it down to the openFinish method within HttpClient? class. I wrote a little test program that creates the threads and opens the URL, without reading the content: ... auto client = new HttpClient(HttpClient.Get, addr); auto response = client.open(); response.close(); client.close(); ... This code worked fine so far when all the functionality from openFinish was removed. Does anybody have an idea, where I should look? I'm not sure, but it seems to be a problem with the line iterator. ciao Lars

Author	Message
lars_kirchhoff Joined: 05/27/08 Posts: 59	Posted: 08/13/09 14:10:04 -- Modified: 08/13/09 14:11:20 by lars_kirchhoff The problem seems to be, that too many connections stay open after the program is running. I'm guessing that some sockets are not properly closed in case of an exception. I will look further I try to pin down the problem.. /lars
lars_kirchhoff Joined: 05/27/08 Posts: 59	Posted: 08/14/09 07:13:04 I just issued a ticket.. /lars