Always adding Computer for master as a fallback
The original proposed fix for JENKINS-7291 creates a Computer object
transitively. This seems unwise as it violates the design of Computer
as stated in the javadoc, and for example we can end up creating two
Computers for the master.
I think a better fix is to create a Computer for the master all the
time, even if there's no executors configured. The discrimination in
Queue.makeBuildable would ensure that such phantom Computer is only used
as a last resort (statistically speaking).
I've also tweaked executors.jelly a bit. I simplified it somewhat
based on the idea that "if there's only one computer to show, the
context is likely making it obvious".
(I must be missing the intricacy in the current code.)
Originally developed in a branch at
2c5b57fcc1 then squashed for clarity.
https://issues.jenkins-ci.org/browse/JENKINS-16942
PrintWriter.println() uses the OS specific line separator. When
SecretRewriter rewrites the secret file, the lines are converted to the
format of the running OS. The test case does not account for this.
Since the test case is coded expecting line endings of \n, this fix
converts \r\n to \n before comparing.
Fixed the throughput problem between master/slave communication.
This fix contains two independent problems.
One was in the remoting. During a large sustained data transfer
(such as artifact archiving and large test reports), the way we
were doing flow control and ACK-ing were penalizing us badly.
I improved the flow control algorithm in remoting 1.23, and also
increased advertised window size so that the transfer can saturate
available bandwidth even when a latency is large. (And unless
the reader side is excessivesly slow, this shouldn't increase
any memory consumption.)
The other fix was in trilead-ssh2, which is our SSH client
implementation used by ssh-slaves plugin. The buffer size for flow
control was too small. I improved the way buffering is done to reduce
the memory footprint when the reader closely follows the writer, then I
increased the advertised window size. Again, this shouldn't increase
memory consumption (in fact it'll likely actually reduce them) unless
the reader end gets abandoned.
On my simulated latency-injected network, the sustained transfer rate is
now on par with scp. We win for smaller files because of the TCP slow
start penality that scp would incur, and we lose a bit as files get
larger due to additional framing overhead.
If you have manually extracted slave.jar and placed them on slaves, you
need to update them to 2.23 to see the performance benefits.
Prevents us from leaving behind a temp dir with cyclic symlinks, which breaks JGit’s clean function.
That in turn can cause test builds to break:
https://jenkins.ci.cloudbees.com/job/core/job/jenkins_main_trunk/783/console
Cleaning workspace
FATAL: Could not delete file …/workspace/core/jenkins_main_trunk/core/target/hudson5309354136960541268tmp/c/symlink/c/symlink/c/symlink/…/c/foo.xml
org.eclipse.jgit.api.errors.JGitInternalException: Could not delete file …/workspace/core/jenkins_main_trunk/core/target/hudson5309354136960541268tmp/c/symlink/c/symlink/c/symlink/…/c/foo.xml
at org.eclipse.jgit.api.CleanCommand.call(CleanCommand.java:137)
at org.jenkinsci.plugins.gitclient.JGitAPIImpl.clean(JGitAPIImpl.java:305)
at hudson.plugins.git.GitAPI.clean(GitAPI.java:248)
at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1213)
at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1196)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2236)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: Could not delete file …/workspace/core/jenkins_main_trunk/core/target/hudson5309354136960541268tmp/c/symlink/c/symlink/c/symlink/…/c/foo.xml
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:142)
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:126)
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:126)
…
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:126)
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:126)
at org.eclipse.jgit.api.CleanCommand.call(CleanCommand.java:132)
... 14 more