Re: [jetty-users] SolrJ/Solr: HTTP protocol violation: Authentication ch

[Shawn Heisey <eclipse@xxxxxxxxxxxx>]

On 5/31/23 02:50, Bill Ross wrote:
> Minimal relevant experience here, but lots of debugging, and since 
> you're stuck:
>
> Is this failing only on the last request of a series of any length?

It's failing in the middle of a run.  My program grabs results from one 
Solr cluster and indexes them into another Solr cluster.  30.2 million 
documents total, it's failing after about 20-25 million docs.

I broke up the migration into six smaller chunks that are all run 
separately (the biggest of which is 9.3 million docs) and now it 
completes with no problem.

I did not see any usable error messages in the Solr server logs.

The documents are indexed in batches of 1000 docs, so it's just slightly 
north of 30000 total requests for the full run.

The way the program works is that I have a query thread querying the 
source cluster and pushing the documents one by one onto a queue.  Then 
the indexing threads pull documents off the queue, organize them into 
batches, and send the batches to the target cluster.

It's being done with SolrJ -- the Java Solr client.  The specfic clients 
I am using utilize the Jetty httpclient, version 10.0.13.

The Solr servers are running a slightly older release that fires up 
Jetty 9.4.48 to start the Solr webapp.

There is a Solr upgrade available that upgrades the Jetty server to 
10.0.13, but the client is resisting an upgrade.

At least I found a way to get the work done by breaking it into six 
smaller runs instead of doing the whole job at once.  Big bonus ... it 
also runs about three times as fast for some reason.  I don't know why, 
but I'll take it!

I'm trying to find out whether the problem is in Jetty (client or 
server), SolrJ, the Solr servers, or my usage.  The particulars make me 
think its something in Jetty, but that could be completely wrong.

Thanks,
Shawn