This service has been turned off.

ssh -p 224 fisheye@fisheye.jenkins-ci.org

Services that run

This host runs FishEye/Crucible and MySQL, which is used as the backend for FishEye.

Starting/stopping FishEye

/etc/init.d/fisheye [start|stop|restart]

Administering FishEye

FishEye admin password is in ~/password.txt of the fisheye user.

Administering MySQL

MySQL has fisheye database accessible by the fisheye user. The password is the same as FishEye admin password.

Incident Records

Hang 2011 Aug 30

Heap dump indicates oldgen & permgen full. Strangely, there's no attempt to grow permgen to the alloted 256MB. Obtained heap dump, modified the boot script to enable the JMX remote port, and restarted.

For whatever reason, remote JMX connectoin isn't working, but XAUTHORITY=/home/kohsuke/.Xauthority sudo jconsole seems to let me run jconsole remote and forward the display. This incident is filed as FSH-6709

root@cucumber:/var/log/apache2/fisheye.jenkins-ci.org# jmap -heap 24386
Attaching to process ID 24386, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 19.1-b02

using thread-local object allocation.
Parallel GC with 2 thread(s)

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 1073741824 (1024.0MB)
   NewSize          = 1310720 (1.25MB)
   MaxNewSize       = 134217728 (128.0MB)
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 268435456 (256.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 44826624 (42.75MB)
   used     = 44826624 (42.75MB)
   free     = 0 (0.0MB)
   100.0% used
From Space:
   capacity = 42074112 (40.125MB)
   used     = 0 (0.0MB)
   free     = 42074112 (40.125MB)
   0.0% used
To Space:
   capacity = 43384832 (41.375MB)
   used     = 0 (0.0MB)
   free     = 43384832 (41.375MB)
   0.0% used
PS Old Generation
   capacity = 939524096 (896.0MB)
   used     = 939479216 (895.9571990966797MB)
   free     = 44880 (0.0428009033203125MB)
   99.99522311346871% used
PS Perm Generation
   capacity = 120127488 (114.5625MB)
   used     = 119802960 (114.25300598144531MB)
   free     = 324528 (0.3094940185546875MB)
   99.729847010536% used

Hang 2011 Aug 27

Heap dump indicates permgen full. Strangely, ps indicates that VM was running with MaxPermSize=256m. It appears that the boot sequence uses a script to tweak FISHEYE_OPTS and ignores our value:

cucumber% sudo jmap -heap 16168    
Attaching to process ID 16168, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 19.1-b02

using thread-local object allocation.
Parallel GC with 2 thread(s)

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 1073741824 (1024.0MB)
   NewSize          = 1310720 (1.25MB)
   MaxNewSize       = 134217728 (128.0MB)
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 268435456 (256.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 80740352 (77.0MB)
   used     = 65356520 (62.328834533691406MB)
   free     = 15383832 (14.671165466308594MB)
   80.94653835544338% used
From Space:
   capacity = 6160384 (5.875MB)
   used     = 2162896 (2.0626983642578125MB)
   free     = 3997488 (3.8123016357421875MB)
   35.10975939162234% used
To Space:
   capacity = 5963776 (5.6875MB)
   used     = 0 (0.0MB)
   free     = 5963776 (5.6875MB)
   0.0% used
PS Old Generation
   capacity = 939524096 (896.0MB)
   used     = 690782352 (658.7813873291016MB)
   free     = 248741744 (237.21861267089844MB)
   73.5247084072658% used
PS Perm Generation
   capacity = 119603200 (114.0625MB)
   used     = 119478504 (113.9435806274414MB)
   free     = 124696 (0.11891937255859375MB)
   99.89574191994863% used

$ ps -ef | grep fisheye
fisheye  16168     1  7 Aug09 ?        1-08:21:22 /usr/bin/java -Xmx1024m -XX:MaxNewSize=128m -XX:MaxPermSize=256m -Dfisheye.library.path= -Dfisheye.inst=/srv/fisheye/FISHEYE_INST -Djava.awt.headless=true -Djava.endorsed.dirs=/srv/fisheye/current/lib/endorsed -jar /srv/fisheye/current/fisheyeboot.jar start

Jenkins Infrastructure : fisheye.jenkins-ci.org