archive-com.com » COM » O » ORCAWARE.COM

Total: 284

Choose link from "Titles, links and description words view":

Or switch to "Titles and links view".

  • with disk peak When WATCH DISK is defined now print each disk s run percent Add a new define WATCH MOUNTS which reports each local mount point s disk space and inode capacity usage available for non root users and percent used This comes from Duncan Lawie tyger hoopoes com Add some smarts so that if the number of interfaces physical disks or mounted partitions changes then a new header is printed This will prevent column name and data mixups when the system configuration changes Version 1 12 Sep 14 1999 Add the page scan rate as scanrate in measure cpu Version 1 11 Aug 13 1999 Add the number of CPUs as ncpus Move measure disk and measure ram sooner in the list of subsystems to handle Increase the number of characters for each network interface from four to five Add new disk reads writes Kbytes read and Kbytes written per second Add number of bytes of free memory in bytes as freememK Version 1 10 Jul 28 1999 Measure the process spawn rate if WATCH CPU is defined and the user is root Version 1 9 Jun 2 1999 If WATCH YAHOO is defined then process the access log as a Yahoo style access log Restructure the code to handle different web server access log formats Version 1 8 Jun 1 1999 If the environmental variable WEB SERVER is defined use its value of the as the name of the process to count for the number of web servers on the system If WEB SERVER is not defined then count number of httpd s Version 1 7 Mar 25 1999 Simplify and speed up count proc by 20 Version 1 6 Feb 23 1999 Print pvm user time and system time correctly Version 1 5 Feb 23 1999 Always write header to a new file Version 1 4 Feb 19 1999 Handle missing HTTP 1 x in access log Version 1 3 Feb 18 1999 On busy machines httpops5 will be enlarged Version 1 2 Feb 18 1999 Output data on integral multiples of interval Version 1 1 Feb 18 1999 Integrate Squid log processing from SE 3 1 Version 1 0 Sep 9 1998 Initial version The default sampling interval in seconds define SAMPLE INTERVAL 300 The maximum number of colums of data define MAX COLUMNS 2048 Define the different parts of the system you want to examine ifdef WATCH OS define WATCH CPU 1 define WATCH MUTEX 1 define WATCH NET 1 define WATCH TCP 1 define WATCH NFS CLIENT 1 define WATCH NFS SERVER 1 define WATCH MOUNTS 1 define USE RAWDISK 1 define WATCH DISK 1 define WATCH DNLC 1 define WATCH INODE 1 define WATCH RAM 1 define WATCH PAGES 1 endif Keep backwards compatibility with WATCH HTTPD ifdef WATCH HTTPD define WATCH WEB 1 endif Keep backwards compatibility with WATCH NFS ifdef WATCH NFS ifndef WATCH NFS CLIENT define WATCH NFS CLIENT 1 endif endif include include include include include include include include include include include include include include include ifdef WATCH MOUNTS include include endif if WATCH CPU WATCH WEB include ifdef WATCH CPU This is the maximum pid on Solaris hosts define DEFAULT MAXPID 30000 include endif ifdef WATCH WEB include Define this macro which returns the size index for a file of a particular size This saves the overhead of a function call define WWW SIZE INDEX size size index if size 1M long dwnld totalz Total size counted from log if WATCH PROXY WATCH SQUID WATCH YAHOO If we re watching a Yahoo log then take the transfer time to be the processing time double www dwnld time sum Transfer time double www dwnld time by size 5 Mean transfer time by size bin endif if WATCH PROXY WATCH SQUID long prxy squid indirect hits that go via PROXY SOCKS parent long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif RAWDISK BEGIN ifdef USE RAWDISK include This extention accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int i int j 0 char long name 16 char short name 16 SE for 2 6 2 7 appears to have a bug where MAX DISK is too large when an A1000 raid controller is present It s not really SE as iostat had to be patched for the same bug The bug appears to be tied to a failure to update the kstat interface diskinfo uses to map short names to long names Since raw disk update has already identified how many physical devices are present and they are listed first we limit our self to mapping the first RAW disk count device name for i 0 i nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK END Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong ofile File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column int current column 0 The current column int print header 1 Flag to flush header Send the stored columns of information to the output print columns string data int i for i 0 i MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column flush output if print header 0 print columns col comment print header 0 print columns col data current column 0 Sets ofile to the output file pointer Creates or appends to the log file if OUTDIR is set otherwise sets the file pointer to STDOUT It start a new log file each day It compresses the previous days log file if the environmental variable COMPRESSOR is set checkoutput tm t now string outdir getenv OUTDIR string outname tm t then char tm buf 32 if outdir nil No output dir so use stdout if ofile 0 First time so print header and set ofile ofile stdout print header 1 return Maintain daily output logfiles in OUTDIR if now tm yday then tm yday First time or day has changed start new logfile if ofile 0 Close and optionally compress the existing output file fclose ofile if compress nil system sprintf compress outname strftime tm buf sizeof tm buf Y m d now outname sprintf s percol s outdir tm buf Open for append either way ofile fopen outname a if ofile 0 perror can t open output logfile exit 1 Always write header print header 1 then now int main int argc string argv utsname t u 1 long now long sleep till Time to sleep to tm t tm now Get the nodename of the machine uname u nodename u 0 nodename program name argv 0 Handle the command line arguments switch argc case 1 break case 2 interval atoi argv 1 break default fprintf stderr usage se Defines s interval n program name fprintf stderr The default interval is d seconds n SAMPLE INTERVAL fprintf stderr s uses the following environmental variables n program name fprintf stderr OUTDIR directory to write log files output to stdout if not set n fprintf stderr WEB SERVER string to search for number of web servers i e httpd n fprintf stderr WEB LOG location of web server access log i e httpd logs access n fprintf stderr GATEWAY special address to monitor i e some where com n fprintf stderr SEARCHURL match for search scripts default is search cgi n fprintf stderr COMPRESSOR compress previous days logs with this command i e gzip 9 n fprintf stderr Add these defines to enable monitoring of specific subsystems n fprintf stderr DWATCH WEB watch web server access logs n fprintf stderr DWATCH PROXY use WEB LOG as a NCSA style proxy log n fprintf stderr DWATCH SQUID use WEB LOG as a Squid log n fprintf stderr DWATCH OS includes all of the below n fprintf stderr DWATCH CPU watch the cpu load run queue etc n fprintf stderr DWATCH MUTEX watch the number of mutex spins n fprintf stderr DWATCH NET watch all Ethernet interfaces n fprintf stderr DWATCH TCP watch all the TCP IP stack n fprintf stderr DWATCH NFS CLIENT watch NFS client requests n fprintf stderr DWATCH NFS SERVER watch NFS server requests n fprintf stderr DWATCH MOUNTS watch usage of mount points n fprintf stderr DWATCH DISK watch disk read write usage n fprintf stderr DWATCH DNLC watch the directory name lookup cache n fprintf stderr DWATCH INODE watch the inode cache n fprintf stderr DWATCH RAM watch memory usage n fprintf stderr DWATCH PAGES watch where pages are allocated n exit 1 break Initialize the various structures initialize Run forever If WATCH WEB is defined then have measure web do the sleeping while it is watching the access log file until the next update time for the whole operating system Also collect the data from the access log file before printing any output for Calculate the next time to sleep to that is an integer multiple of the interval time Make sure that at least half of the interval passes before waking up now time 0 sleep till now interval interval while sleep till mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in the long interval time span use the difference between the previous and current mpid over a 5 second interval to calculate the long interval difference mpid current mpid5 diff mpid now mpid5 now else sleep sleep till now else sleep sleep till now endif now time 0 ifdef WATCH CPU measure cpu p vmstat pvm double mpid interval double mpid rate pvm vmglobal total In SE 3 0 and below user time and system time are int and in SE 3 1 and above they are double so cast everything to double using 0 0 put output usr sprintf 5 1f pvm user time 0 0 put output sys sprintf 5 1f pvm system time 0 0 put output wio sprintf 5 1f pvm wait time 0 0 put output idle sprintf 5 1f pvm idle time 0 0 put output 1runq sprintf 6 2f tmp kstat misc avenrun 1min 256 0 put output 5runq sprintf 6 2f tmp kstat misc avenrun 5min 256 0 put output 15runq sprintf 6 2f tmp kstat misc avenrun 15min 256 0 put output proc sprintf 5lu tmp kstat misc nproc put output runque sprintf 8 2f pvm runque 0 0 put output waiting sprintf 8 2f pvm waiting 0 0 put output swpque sprintf 8 2f pvm swpque 0 0 put output scanrate sprintf 8 3f pvm scan 0 0 Calculate the rate of new process spawning if can read kernel 0 mpid interval mpid now mpid then 0 000000001 mpid rate mpid current mpid previous mpid interval put output proc s sprintf 7 3f mpid rate put output proc p5s sprintf 9 4f mpid5 rate Reset counters mpid then mpid now mpid previous mpid current mpid5 rate 0 endif ifdef WATCH MUTEX measure mutex put output smtx sprintf 5d tmp mutex smtx put output smtx cpu sprintf 8d tmp mutex smtx GLOBAL pvm ncpus endif ifdef WATCH NET measure net int previous count 1 int current count int i current count 0 for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a very valid stat anyway What hapens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still geting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads RAW disk i reads total writes RAW disk i writes total readk RAW disk i kreads total writek RAW disk i kwrites mean disk busy RAW disk i run percent if RAW disk i run percent peak disk busy peak disk busy RAW disk i run percent ifdef HAVE EMC DISK CONTROL endif else for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a very valid stat anyway What hapens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still geting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads GLOBAL disk i reads total writes GLOBAL disk i writes total readk GLOBAL disk i kreads total writek GLOBAL disk i kwrites mean disk busy GLOBAL disk i run percent if GLOBAL disk i run percent peak disk busy peak disk busy GLOBAL disk i run percent ifdef HAVE EMC DISK CONTROL endif endif if disk count 0 mean disk busy mean disk busy disk count put output disk peak sprintf 9 3f peak disk busy put output disk mean sprintf 9 3f mean disk busy put output disk rd s sprintf 9 1f total reads put output disk wr s sprintf 9 1f total writes put output disk rK s sprintf 9 1f total readk put output disk wK s sprintf 9 1f total writek If the number of disks has changed say

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.28b6.txt (2016-04-24)
    Open archived version from archive



  • long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif RAWDISK BEGIN ifdef USE RAWDISK include This extension accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int i int j 0 char long name 16 char short name 16 SE for 2 6 2 7 appears to have a bug where MAX DISK is too large when an A1000 raid controller is present It s not really SE as iostat had to be patched for the same bug The bug appears to be tied to a failure to update the kstat interface diskinfo uses to map short names to long names Since raw disk update has already identified how many physical devices are present and they are listed first we limit our self to mapping the first RAW disk count device name for i 0 i nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK END Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong ofile File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column string col previous comment MAX COLUMNS Previous comments int current column 0 The current column int previous number columns 1 Number columns printed last int print header 1 Flag to flush header This is a list of the extensions the compress programs add to the compress filename define NUMBER COMPRESS SUFFIXES 3 string compression suffixes NUMBER COMPRESS SUFFIXES Z gz bz2 Add one column of comments and data to the buffers put output string comment string data if current column MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column Send the stored columns of information to the output print columns string data int i for i 0 i mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in the long interval time span use the difference between the previous and current mpid over a 5 second interval to calculate the long interval difference mpid current mpid5 diff mpid now mpid5 now else sleep sleep till now else sleep sleep till now endif now time 0 ifdef WATCH

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.28b7.txt (2016-04-24)
    Open archived version from archive


  • Yahoo log then take the transfer time to be the processing time double www dwnld time sum Transfer time double www dwnld time by size 5 Mean transfer time by size bin endif if WATCH PROXY WATCH SQUID long prxy squid indirect hits that go via PROXY SOCKS parent long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif RAWDISK BEGIN ifdef USE RAWDISK include This extension accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int i int j 0 char long name 16 char short name 16 SE for 2 6 2 7 appears to have a bug where MAX DISK is too large when an A1000 raid controller is present It s not really SE as iostat had to be patched for the same bug The bug appears to be tied to a failure to update the kstat interface diskinfo uses to map short names to long names Since raw disk update has already identified how many physical devices are present and they are listed first we limit our self to mapping the first RAW disk count device name for i 0 i nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK END Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong ofile File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column string col previous comment MAX COLUMNS Previous comments int current column 0 The current column int previous number columns 1 Number columns printed last int print header 1 Flag to flush header This is a list of the extensions the compress programs add to the compress filename define NUMBER COMPRESS SUFFIXES 3 string compression suffixes NUMBER COMPRESS SUFFIXES Z gz bz2 Add one column of comments and data to the buffers put output string comment string data if current column MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column Send the stored columns of information to the output print columns string data int i for i 0 i mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.29.txt (2016-04-24)
    Open archived version from archive


  • The sum of v 2 3 reads writes will be less than nfss calls as the other types of NFS calls such as getattr and lookup are not included Contributed by Paul Haldane This code is enabled by the standard DWATCH OS or individually by DWATCH NFS SERVER The define DWATCH NFS has been superseded by DWATCH NFS CLIENT but to keep backwards compatibility DWATCH NFS CLIENT will be defined if DWATCH NFS is defined Version 1 21 Jan 12 2000 Prevent core dumps on extremely long access log lines Version 1 20 Oct 20 1999 Update my email address Version 1 19 Oct 13 1999 Prevent a division by zero in calculating the mean disk busy if the number of disks on the system is 0 Version 1 18 Oct 12 1999 Rename disk runp c t d to disk runp c t d to remove the s Version 1 17 Oct 8 1999 Do not record mount point statistics for locally mounted cdrom partitions Version 1 16 Oct 7 1999 To keep backwards compatibility define WATCH WEB if WATCH HTTPD is defined If the COMPRESSOR environmental variable is defined then when a new log file is opened for a new day the just closed log file is compressed using the COMPRESSOR command in the following manner system sprintf s s COMPRESSOR log file COMPRESSOR should be set to something like gzip 9 or compress or bzip2 9 Version 1 15 Oct 5 1999 kvm mpid is a int not a long This caused problems on Solaris 7 hosts running a 64 bit kernel Version 1 14 Oct 1 1999 Rename disk c t d column names to disk runp c t d to better reflect the data being recorded and to allow for more per disk information later Version 1 13 Sep 24 1999 Fix a bug in the disk mean calculation where it was being divided by the wrong disk count Now it should be much larger and in scale with disk peak When WATCH DISK is defined now print each disk s run percent Add a new define WATCH MOUNTS which reports each local mount point s disk space and inode capacity usage available for non root users and percent used This comes from Duncan Lawie tyger hoopoes com Add some smarts so that if the number of interfaces physical disks or mounted partitions changes then a new header is printed This will prevent column name and data mixups when the system configuration changes Version 1 12 Sep 14 1999 Add the page scan rate as scanrate in measure cpu Version 1 11 Aug 13 1999 Add the number of CPUs as ncpus Move measure disk and measure ram sooner in the list of subsystems to handle Increase the number of characters for each network interface from four to five Add new disk reads writes Kbytes read and Kbytes written per second Add number of bytes of free memory in bytes as freememK Version 1 10 Jul 28 1999 Measure the process spawn rate if WATCH CPU is defined and the user is root Version 1 9 Jun 2 1999 If WATCH YAHOO is defined then process the access log as a Yahoo style access log Restructure the code to handle different web server access log formats Version 1 8 Jun 1 1999 If the environmental variable WEB SERVER is defined use its value of the as the name of the process to count for the number of web servers on the system If WEB SERVER is not defined then count number of httpd s Version 1 7 Mar 25 1999 Simplify and speed up count proc by 20 Version 1 6 Feb 23 1999 Print pvm user time and system time correctly Version 1 5 Feb 23 1999 Always write header to a new file Version 1 4 Feb 19 1999 Handle missing HTTP 1 x in access log Version 1 3 Feb 18 1999 On busy machines httpops5 will be enlarged Version 1 2 Feb 18 1999 Output data on integral multiples of interval Version 1 1 Feb 18 1999 Integrate Squid log processing from SE 3 1 Version 1 0 Sep 9 1998 Initial version The default sampling interval in seconds define SAMPLE INTERVAL 300 The maximum number of columns of data define MAX COLUMNS 2048 Define the different parts of the system you want to examine ifdef WATCH OS define WATCH CPU 1 define WATCH MUTEX 1 define WATCH NET 1 define WATCH TCP 1 define WATCH NFS CLIENT 1 define WATCH NFS SERVER 1 define WATCH MOUNTS 1 define USE RAWDISK 1 define WATCH DISK 1 define WATCH DNLC 1 define WATCH INODE 1 define WATCH RAM 1 define WATCH PAGES 1 endif Keep backwards compatibility with WATCH HTTPD ifdef WATCH HTTPD define WATCH WEB 1 endif Keep backwards compatibility with WATCH NFS ifdef WATCH NFS ifndef WATCH NFS CLIENT define WATCH NFS CLIENT 1 endif endif include include include include include include include include include include include include include include include include ifdef WATCH MOUNTS include include endif if WATCH CPU WATCH WEB include ifdef WATCH CPU This is the maximum pid on Solaris hosts define DEFAULT MAXPID 30000 include endif ifdef WATCH WEB Define this macro which returns the size index for a file of a particular size This saves the overhead of a function call define WWW SIZE INDEX size size index if size 1M long dwnld totalz Total size counted from log if WATCH PROXY WATCH SQUID WATCH YAHOO If we re watching a Yahoo log then take the transfer time to be the processing time double www dwnld time sum Transfer time double www dwnld time by size 5 Mean transfer time by size bin endif if WATCH PROXY WATCH SQUID long prxy squid indirect hits that go via PROXY SOCKS parent long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif ifdef USE RAWDISK include This code was developed so that the performance of virtual disks originating from a Sun A1000 raid controller could be monitored These disks do not show up in the GLOBAL disk io structure of SE This extension accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface that matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int first len char first name 16 char long name 16 char short name 16 int i int j This section is used to map short names to long names Since raw disk update has already identified how many physical devices it simply tries to find theses devices in GLOBAL disk info SE appears to have a bug where GLOBAL diskinfo size can be larger than the number of entries in GLOBAL disk info under a variety of conditions In later versions of SE GLOBAL diskinfo size has been removed This appears to fix the above problem This code uses MAX RAWDISKS for the table length and the assumption that short disks names come before short disk partition names to detect the end of the table If it fails to detect the end it will coredump when it addresses unallocated memory strcpy first name GLOBAL disk info 0 short name first len strlen first name for i 0 i 0 if strncmp first name short name first len 0 break if strcmp RAW disk i short name short name 0 strcpy long name GLOBAL disk info j long name strcpy RAW disk i long name long name break RAW disk map 0 raw disk update int rdisk ulong ul kstat ctl t kc 1 kstat t kp 1 kstat t nkp 1 kstat io t kio ulonglong nread ulonglong nwritten uint reads uint writes longlong wtime longlong wlentime longlong wlastupdate longlong rtime longlong rlentime longlong rlastupdate longlong wcnt longlong rcnt double read writes double big etime double elapsed etime double hz etime double nanosecond NANOSEC double update double delta timeval t time update 1 ulong time void char short name 8 gettimeofday time update time void update time update 0 tv sec time update 0 tv usec 1000000 0 delta update RAW disk lastupdate RAW disk lastupdate update kc 0 kstat open Read them if kstat read kc kp 0 1 perror raw disk update kstat read exit 1 Traverse the chain looking for IO events for ul kc 0 kc chain ul 0 ul nkp 0 ks next struct fill nkp 0 ul if nkp 0 ks type KSTAT TYPE IO strcpy short name nkp 0 ks name if short name 0 m short name 0 n strchr short name nil Try to locate device for rdisk 0 rdisk nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong ofile File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column string col previous comment MAX COLUMNS Previous comments int current column 0 The current column int previous number columns 1 Number columns printed last int print header 1 Flag to flush header This is a list of the extensions the compress programs add to the compress filename define NUMBER COMPRESS SUFFIXES 3 string compression suffixes NUMBER COMPRESS SUFFIXES Z gz bz2 Add one column of comments and data to the buffers put output string comment string data if current column MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column Send the stored columns of information to the output print columns string data int i for i 0 i mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in the long interval time span use the difference between the previous and current mpid over a 5 second interval to calculate the long interval difference mpid current mpid5 diff mpid now mpid5 now else sleep sleep till now else sleep sleep till now endif now time 0 ifdef WATCH CPU measure cpu p vmstat pvm double mpid interval double mpid rate pvm vmglobal total In SE 3 0 and below user time and system time are int and in SE 3 1 and above they are double so cast everything to double using 0 0 put output usr sprintf 5 1f pvm user time 0 0 put output sys sprintf 5 1f pvm system time 0 0 put output wio sprintf 5 1f pvm wait time 0 0 put output idle sprintf 5 1f pvm idle time 0 0 put output 1runq sprintf 6 2f tmp kstat misc avenrun 1min 256 0 put output 5runq sprintf 6 2f tmp kstat misc avenrun 5min 256 0 put output 15runq sprintf 6 2f tmp kstat misc avenrun 15min 256 0 put output proc sprintf 5lu tmp kstat misc nproc put output runque sprintf 8 2f pvm runque 0 0 put output waiting sprintf 8 2f pvm waiting 0 0 put output swpque sprintf 8 2f pvm swpque 0 0 put output scanrate sprintf 8 3f pvm scan 0 0 Calculate the rate of new process spawning if can read kernel 0 mpid interval mpid now mpid then 0 000000001 mpid rate mpid current mpid previous mpid interval put output proc s sprintf 7 3f mpid rate put output proc p5s sprintf 9 4f mpid5 rate Reset counters mpid then mpid now mpid previous mpid current mpid5 rate 0 endif ifdef WATCH MUTEX measure mutex put output smtx sprintf 5d tmp lr mutex smtx put output smtx cpu sprintf 8d tmp lr mutex smtx GLOBAL pvm ncpus endif ifdef WATCH NET measure net int previous count 1 int current count int i current count 0 for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads RAW disk i reads total writes RAW disk i writes total readk RAW disk i kreads total writek RAW disk i kwrites mean disk busy RAW disk i run percent if RAW disk i run percent peak disk busy peak disk busy RAW disk i run percent ifdef HAVE EMC DISK CONTROL endif else for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads GLOBAL disk i reads total writes GLOBAL disk i writes total readk GLOBAL disk i kreads total writek GLOBAL disk i kwrites mean disk busy GLOBAL disk i run percent if GLOBAL disk i run percent peak disk busy peak disk busy GLOBAL disk i run percent ifdef HAVE EMC DISK CONTROL endif endif if disk count 0 mean disk busy mean disk busy disk count put output disk peak

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.30.txt (2016-04-24)
    Open archived version from archive


  • NFS CLIENT but to keep backwards compatibility DWATCH NFS CLIENT will be defined if DWATCH NFS is defined Version 1 21 Jan 12 2000 Prevent core dumps on extremely long access log lines Version 1 20 Oct 20 1999 Update my email address Version 1 19 Oct 13 1999 Prevent a division by zero in calculating the mean disk busy if the number of disks on the system is 0 Version 1 18 Oct 12 1999 Rename disk runp c t d to disk runp c t d to remove the s Version 1 17 Oct 8 1999 Do not record mount point statistics for locally mounted cdrom partitions Version 1 16 Oct 7 1999 To keep backwards compatibility define WATCH WEB if WATCH HTTPD is defined If the COMPRESSOR environmental variable is defined then when a new log file is opened for a new day the just closed log file is compressed using the COMPRESSOR command in the following manner system sprintf s s COMPRESSOR log file COMPRESSOR should be set to something like gzip 9 or compress or bzip2 9 Version 1 15 Oct 5 1999 kvm mpid is a int not a long This caused problems on Solaris 7 hosts running a 64 bit kernel Version 1 14 Oct 1 1999 Rename disk c t d column names to disk runp c t d to better reflect the data being recorded and to allow for more per disk information later Version 1 13 Sep 24 1999 Fix a bug in the disk mean calculation where it was being divided by the wrong disk count Now it should be much larger and in scale with disk peak When WATCH DISK is defined now print each disk s run percent Add a new define WATCH MOUNTS which reports each local mount point s disk space and inode capacity usage available for non root users and percent used This comes from Duncan Lawie tyger hoopoes com Add some smarts so that if the number of interfaces physical disks or mounted partitions changes then a new header is printed This will prevent column name and data mixups when the system configuration changes Version 1 12 Sep 14 1999 Add the page scan rate as scanrate in measure cpu Version 1 11 Aug 13 1999 Add the number of CPUs as ncpus Move measure disk and measure ram sooner in the list of subsystems to handle Increase the number of characters for each network interface from four to five Add new disk reads writes Kbytes read and Kbytes written per second Add number of bytes of free memory in bytes as freememK Version 1 10 Jul 28 1999 Measure the process spawn rate if WATCH CPU is defined and the user is root Version 1 9 Jun 2 1999 If WATCH YAHOO is defined then process the access log as a Yahoo style access log Restructure the code to handle different web server access log formats Version 1 8 Jun 1 1999 If the environmental variable WEB SERVER is defined use its value of the as the name of the process to count for the number of web servers on the system If WEB SERVER is not defined then count number of httpd s Version 1 7 Mar 25 1999 Simplify and speed up count proc by 20 Version 1 6 Feb 23 1999 Print pvm user time and system time correctly Version 1 5 Feb 23 1999 Always write header to a new file Version 1 4 Feb 19 1999 Handle missing HTTP 1 x in access log Version 1 3 Feb 18 1999 On busy machines httpops5 will be enlarged Version 1 2 Feb 18 1999 Output data on integral multiples of interval Version 1 1 Feb 18 1999 Integrate Squid log processing from SE 3 1 Version 1 0 Sep 9 1998 Initial version The default sampling interval in seconds define SAMPLE INTERVAL 300 The maximum number of columns of data define MAX COLUMNS 2048 Define the different parts of the system you want to examine ifdef WATCH OS define WATCH CPU 1 define WATCH MUTEX 1 define WATCH NET 1 define WATCH TCP 1 define WATCH NFS CLIENT 1 define WATCH NFS SERVER 1 define WATCH MOUNTS 1 define USE RAWDISK 1 define WATCH DISK 1 define WATCH DNLC 1 define WATCH INODE 1 define WATCH RAM 1 define WATCH PAGES 1 endif Keep backwards compatibility with WATCH HTTPD ifdef WATCH HTTPD define WATCH WEB 1 endif Keep backwards compatibility with WATCH NFS ifdef WATCH NFS ifndef WATCH NFS CLIENT define WATCH NFS CLIENT 1 endif endif include include include include include include include include include include include include include include include include ifdef WATCH MOUNTS include include endif if WATCH CPU WATCH WEB include ifdef WATCH CPU This is the maximum pid on Solaris hosts define DEFAULT MAXPID 30000 include endif ifdef WATCH WEB Define this macro which returns the size index for a file of a particular size This saves the overhead of a function call define WWW SIZE INDEX size size index if size 1M long dwnld totalz Total size counted from log if WATCH PROXY WATCH SQUID WATCH YAHOO If we re watching a Yahoo log then take the transfer time to be the processing time double www dwnld time sum Transfer time double www dwnld time by size 5 Mean transfer time by size bin endif if WATCH PROXY WATCH SQUID long prxy squid indirect hits that go via PROXY SOCKS parent long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif ifdef USE RAWDISK include This code was developed so that the performance of virtual disks originating from a Sun A1000 raid controller could be monitored These disks do not show up in the GLOBAL disk io structure of SE This extension accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface that matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int first len char first name 16 char long name 16 char short name 16 int i int j This section is used to map short names to long names Since raw disk update has already identified how many physical devices it simply tries to find theses devices in GLOBAL disk info SE appears to have a bug where GLOBAL diskinfo size can be larger than the number of entries in GLOBAL disk info under a variety of conditions In later versions of SE GLOBAL diskinfo size has been removed This appears to fix the above problem This code uses MAX RAWDISKS for the table length and the assumption that short disks names come before short disk partition names to detect the end of the table If it fails to detect the end it will coredump when it addresses unallocated memory strcpy first name GLOBAL disk info 0 short name first len strlen first name for i 0 i 0 if strncmp first name short name first len 0 break if strcmp RAW disk i short name short name 0 strcpy long name GLOBAL disk info j long name strcpy RAW disk i long name long name break RAW disk map 0 raw disk update int rdisk ulong ul kstat ctl t kc 1 kstat t kp 1 kstat t nkp 1 kstat io t kio ulonglong nread ulonglong nwritten uint reads uint writes longlong wtime longlong wlentime longlong wlastupdate longlong rtime longlong rlentime longlong rlastupdate longlong wcnt longlong rcnt double read writes double big etime double elapsed etime double hz etime double nanosecond NANOSEC double update double delta timeval t time update 1 ulong time void char short name 8 gettimeofday time update time void update time update 0 tv sec time update 0 tv usec 1000000 0 delta update RAW disk lastupdate RAW disk lastupdate update kc 0 kstat open Read them if kstat read kc kp 0 1 perror raw disk update kstat read exit 1 Traverse the chain looking for IO events for ul kc 0 kc chain ul 0 ul nkp 0 ks next struct fill nkp 0 ul if nkp 0 ks type KSTAT TYPE IO strcpy short name nkp 0 ks name if short name 0 m short name 0 n strchr short name nil Try to locate device for rdisk 0 rdisk nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong ofile File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column string col previous comment MAX COLUMNS Previous comments int current column 0 The current column int previous number columns 1 Number columns printed last int print header 1 Flag to flush header This is a list of the extensions the compress programs add to the compress filename define NUMBER COMPRESS SUFFIXES 3 string compression suffixes NUMBER COMPRESS SUFFIXES Z gz bz2 Add one column of comments and data to the buffers put output string comment string data if current column MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column Send the stored columns of information to the output print columns string data int i for i 0 i mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in the long interval time span use the difference between the previous and current mpid over a 5 second interval to calculate the long interval difference mpid current mpid5 diff mpid now mpid5 now else sleep sleep till now else sleep sleep till now endif now time 0 ifdef WATCH CPU measure cpu p vmstat pvm double mpid interval double mpid rate pvm vmglobal total In SE 3 0 and below user time and system time are int and in SE 3 1 and above they are double so cast everything to double using 0 0 put output usr sprintf 5 1f pvm user time 0 0 put output sys sprintf 5 1f pvm system time 0 0 put output wio sprintf 5 1f pvm wait time 0 0 put output idle sprintf 5 1f pvm idle time 0 0 put output 1runq sprintf 6 2f tmp kstat misc avenrun 1min 256 0 put output 5runq sprintf 6 2f tmp kstat misc avenrun 5min 256 0 put output 15runq sprintf 6 2f tmp kstat misc avenrun 15min 256 0 put output proc sprintf 5lu tmp kstat misc nproc put output runque sprintf 8 2f pvm runque 0 0 put output waiting sprintf 8 2f pvm waiting 0 0 put output swpque sprintf 8 2f pvm swpque 0 0 put output scanrate sprintf 8 3f pvm scan 0 0 Calculate the rate of new process spawning if can read kernel 0 mpid interval mpid now mpid then 0 000000001 mpid rate mpid current mpid previous mpid interval put output proc s sprintf 7 3f mpid rate put output proc p5s sprintf 9 4f mpid5 rate Reset counters mpid then mpid now mpid previous mpid current mpid5 rate 0 endif ifdef WATCH MUTEX measure mutex put output smtx sprintf 5d tmp mutex smtx put output smtx cpu sprintf 8d tmp mutex smtx GLOBAL pvm ncpus endif ifdef WATCH NET measure net int previous count 1 int current count int i current count 0 for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads RAW disk i reads total writes RAW disk i writes total readk RAW disk i kreads total writek RAW disk i kwrites mean disk busy RAW disk i run percent if RAW disk i run percent peak disk busy peak disk busy RAW disk i run percent ifdef HAVE EMC DISK CONTROL endif else for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads GLOBAL disk i reads total writes GLOBAL disk i writes total readk GLOBAL disk i kreads total writek GLOBAL disk i kwrites mean disk busy GLOBAL disk i run percent if GLOBAL disk i run percent peak disk busy peak disk busy GLOBAL disk i run percent ifdef HAVE EMC DISK CONTROL endif endif if disk count 0 mean disk busy mean disk busy disk count put output disk peak sprintf 9 3f peak disk busy put output disk mean sprintf 9 3f mean disk busy put output disk rd s sprintf 9 1f total reads put output disk wr s sprintf 9 1f total writes put output disk rK s sprintf 9 1f total readk put output disk wK s sprintf 9 1f total writek If the number

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.30b1.txt (2016-04-24)
    Open archived version from archive


  • reads writes will be less than nfss calls as the other types of NFS calls such as getattr and lookup are not included Contributed by Paul Haldane This code is enabled by the standard DWATCH OS or individually by DWATCH NFS SERVER The define DWATCH NFS has been superseded by DWATCH NFS CLIENT but to keep backwards compatibility DWATCH NFS CLIENT will be defined if DWATCH NFS is defined Version 1 21 Jan 12 2000 Prevent core dumps on extremely long access log lines Version 1 20 Oct 20 1999 Update my email address Version 1 19 Oct 13 1999 Prevent a division by zero in calculating the mean disk busy if the number of disks on the system is 0 Version 1 18 Oct 12 1999 Rename disk runp c t d to disk runp c t d to remove the s Version 1 17 Oct 8 1999 Do not record mount point statistics for locally mounted cdrom partitions Version 1 16 Oct 7 1999 To keep backwards compatibility define WATCH WEB if WATCH HTTPD is defined If the COMPRESSOR environmental variable is defined then when a new log file is opened for a new day the just closed log file is compressed using the COMPRESSOR command in the following manner system sprintf s s COMPRESSOR log file COMPRESSOR should be set to something like gzip 9 or compress or bzip2 9 Version 1 15 Oct 5 1999 kvm mpid is a int not a long This caused problems on Solaris 7 hosts running a 64 bit kernel Version 1 14 Oct 1 1999 Rename disk c t d column names to disk runp c t d to better reflect the data being recorded and to allow for more per disk information later Version 1 13 Sep 24 1999 Fix a bug in the disk mean calculation where it was being divided by the wrong disk count Now it should be much larger and in scale with disk peak When WATCH DISK is defined now print each disk s run percent Add a new define WATCH MOUNTS which reports each local mount point s disk space and inode capacity usage available for non root users and percent used This comes from Duncan Lawie tyger hoopoes com Add some smarts so that if the number of interfaces physical disks or mounted partitions changes then a new header is printed This will prevent column name and data mixups when the system configuration changes Version 1 12 Sep 14 1999 Add the page scan rate as scanrate in measure cpu Version 1 11 Aug 13 1999 Add the number of CPUs as ncpus Move measure disk and measure ram sooner in the list of subsystems to handle Increase the number of characters for each network interface from four to five Add new disk reads writes Kbytes read and Kbytes written per second Add number of bytes of free memory in bytes as freememK Version 1 10 Jul 28 1999 Measure the process spawn rate if WATCH CPU is defined and the user is root Version 1 9 Jun 2 1999 If WATCH YAHOO is defined then process the access log as a Yahoo style access log Restructure the code to handle different web server access log formats Version 1 8 Jun 1 1999 If the environmental variable WEB SERVER is defined use its value of the as the name of the process to count for the number of web servers on the system If WEB SERVER is not defined then count number of httpd s Version 1 7 Mar 25 1999 Simplify and speed up count proc by 20 Version 1 6 Feb 23 1999 Print pvm user time and system time correctly Version 1 5 Feb 23 1999 Always write header to a new file Version 1 4 Feb 19 1999 Handle missing HTTP 1 x in access log Version 1 3 Feb 18 1999 On busy machines httpops5 will be enlarged Version 1 2 Feb 18 1999 Output data on integral multiples of interval Version 1 1 Feb 18 1999 Integrate Squid log processing from SE 3 1 Version 1 0 Sep 9 1998 Initial version The default sampling interval in seconds define SAMPLE INTERVAL 300 The maximum number of columns of data define MAX COLUMNS 2048 Define the different parts of the system you want to examine ifdef WATCH OS define WATCH CPU 1 define WATCH MUTEX 1 define WATCH NET 1 define WATCH TCP 1 define WATCH NFS CLIENT 1 define WATCH NFS SERVER 1 define WATCH MOUNTS 1 define USE RAWDISK 1 define WATCH DISK 1 define WATCH DNLC 1 define WATCH INODE 1 define WATCH RAM 1 define WATCH PAGES 1 endif Keep backwards compatibility with WATCH HTTPD ifdef WATCH HTTPD define WATCH WEB 1 endif Keep backwards compatibility with WATCH NFS ifdef WATCH NFS ifndef WATCH NFS CLIENT define WATCH NFS CLIENT 1 endif endif include include include include include include include include include include include include include include include include ifdef WATCH MOUNTS include include endif if WATCH CPU WATCH WEB include ifdef WATCH CPU This is the maximum pid on Solaris hosts define DEFAULT MAXPID 30000 include endif ifdef WATCH WEB Define this macro which returns the size index for a file of a particular size This saves the overhead of a function call define WWW SIZE INDEX size size index if size 1M long dwnld totalz Total size counted from log if WATCH PROXY WATCH SQUID WATCH YAHOO If we re watching a Yahoo log then take the transfer time to be the processing time double www dwnld time sum Transfer time double www dwnld time by size 5 Mean transfer time by size bin endif if WATCH PROXY WATCH SQUID long prxy squid indirect hits that go via PROXY SOCKS parent long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif ifdef USE RAWDISK include This code was developed so that the performance of virtual disks originating from a Sun A1000 raid controller could be monitored These disks do not show up in the GLOBAL disk io structure of SE This extension accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface that matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int first len char first name 16 char long name 16 char short name 16 int i int j This section is used to map short names to long names Since raw disk update has already identified how many physical devices it simply tries to find theses devices in GLOBAL disk info SE appears to have a bug where GLOBAL diskinfo size can be larger than the number of entries in GLOBAL disk info under a variety of conditions In later versions of SE GLOBAL diskinfo size has been removed This appears to fix the above problem This code uses MAX RAWDISKS for the table length and the assumption that short disks names come before short disk partition names to detect the end of the table If it fails to detect the end it will coredump when it addresses unallocated memory strcpy first name GLOBAL disk info 0 short name first len strlen first name for i 0 i 0 if strncmp first name short name first len 0 break if strcmp RAW disk i short name short name 0 strcpy long name GLOBAL disk info j long name strcpy RAW disk i long name long name break RAW disk map 0 raw disk update int rdisk ulong ul kstat ctl t kc 1 kstat t kp 1 kstat t nkp 1 kstat io t kio ulonglong nread ulonglong nwritten uint reads uint writes longlong wtime longlong wlentime longlong wlastupdate longlong rtime longlong rlentime longlong rlastupdate longlong wcnt longlong rcnt double read writes double big etime double elapsed etime double hz etime double nanosecond NANOSEC double update double delta timeval t time update 1 ulong time void char short name 8 gettimeofday time update time void update time update 0 tv sec time update 0 tv usec 1000000 0 delta update RAW disk lastupdate RAW disk lastupdate update kc 0 kstat open Read them if kstat read kc kp 0 1 perror raw disk update kstat read exit 1 Traverse the chain looking for IO events for ul kc 0 kc chain ul 0 ul nkp 0 ks next struct fill nkp 0 ul if nkp 0 ks type KSTAT TYPE IO strcpy short name nkp 0 ks name if short name 0 m short name 0 n strchr short name nil Try to locate device for rdisk 0 rdisk nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong ofile File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column string col previous comment MAX COLUMNS Previous comments int current column 0 The current column int previous number columns 1 Number columns printed last int print header 1 Flag to flush header This is a list of the extensions the compress programs add to the compress filename define NUMBER COMPRESS SUFFIXES 3 string compression suffixes NUMBER COMPRESS SUFFIXES Z gz bz2 Add one column of comments and data to the buffers put output string comment string data if current column MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column Send the stored columns of information to the output print columns string data int i for i 0 i mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in the long interval time span use the difference between the previous and current mpid over a 5 second interval to calculate the long interval difference mpid current mpid5 diff mpid now mpid5 now else sleep sleep till now else sleep sleep till now endif now time 0 ifdef WATCH CPU measure cpu p vmstat pvm double mpid interval double mpid rate pvm vmglobal total In SE 3 0 and below user time and system time are int and in SE 3 1 and above they are double so cast everything to double using 0 0 put output usr sprintf 5 1f pvm user time 0 0 put output sys sprintf 5 1f pvm system time 0 0 put output wio sprintf 5 1f pvm wait time 0 0 put output idle sprintf 5 1f pvm idle time 0 0 put output 1runq sprintf 6 2f tmp kstat misc avenrun 1min 256 0 put output 5runq sprintf 6 2f tmp kstat misc avenrun 5min 256 0 put output 15runq sprintf 6 2f tmp kstat misc avenrun 15min 256 0 put output proc sprintf 5lu tmp kstat misc nproc put output runque sprintf 8 2f pvm runque 0 0 put output waiting sprintf 8 2f pvm waiting 0 0 put output swpque sprintf 8 2f pvm swpque 0 0 put output scanrate sprintf 8 3f pvm scan 0 0 Calculate the rate of new process spawning if can read kernel 0 mpid interval mpid now mpid then 0 000000001 mpid rate mpid current mpid previous mpid interval put output proc s sprintf 7 3f mpid rate put output proc p5s sprintf 9 4f mpid5 rate Reset counters mpid then mpid now mpid previous mpid current mpid5 rate 0 endif ifdef WATCH MUTEX measure mutex put output smtx sprintf 5d tmp lr mutex smtx put output smtx cpu sprintf 8d tmp lr mutex smtx GLOBAL pvm ncpus endif ifdef WATCH NET measure net int previous count 1 int current count int i current count 0 for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads RAW disk i reads total writes RAW disk i writes total readk RAW disk i kreads total writek RAW disk i kwrites mean disk busy RAW disk i run percent if RAW disk i run percent peak disk busy peak disk busy RAW disk i run percent ifdef HAVE EMC DISK CONTROL endif else for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads GLOBAL disk i reads total writes GLOBAL disk i writes total readk GLOBAL disk i kreads total writek GLOBAL disk i kwrites mean disk busy GLOBAL disk i run percent if GLOBAL disk i run percent peak disk busy peak disk busy GLOBAL disk i run percent ifdef HAVE EMC DISK CONTROL endif endif if disk count 0 mean disk busy mean disk busy disk count put output disk peak sprintf 9 3f peak disk busy

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.30b2.txt (2016-04-24)
    Open archived version from archive


  • 2 3 reads writes which are nfss calls broken down into NFS version 2 and NFS version 3 calls The sum of v 2 3 reads writes will be less than nfss calls as the other types of NFS calls such as getattr and lookup are not included Contributed by Paul Haldane This code is enabled by the standard DWATCH OS or individually by DWATCH NFS SERVER The define DWATCH NFS has been superseded by DWATCH NFS CLIENT but to keep backwards compatibility DWATCH NFS CLIENT will be defined if DWATCH NFS is defined Version 1 21 Jan 12 2000 Prevent core dumps on extremely long access log lines Version 1 20 Oct 20 1999 Update my email address Version 1 19 Oct 13 1999 Prevent a division by zero in calculating the mean disk busy if the number of disks on the system is 0 Version 1 18 Oct 12 1999 Rename disk runp c t d to disk runp c t d to remove the s Version 1 17 Oct 8 1999 Do not record mount point statistics for locally mounted cdrom partitions Version 1 16 Oct 7 1999 To keep backwards compatibility define WATCH WEB if WATCH HTTPD is defined If the COMPRESSOR environmental variable is defined then when a new log file is opened for a new day the just closed log file is compressed using the COMPRESSOR command in the following manner system sprintf s s COMPRESSOR log file COMPRESSOR should be set to something like gzip 9 or compress or bzip2 9 Version 1 15 Oct 5 1999 kvm mpid is a int not a long This caused problems on Solaris 7 hosts running a 64 bit kernel Version 1 14 Oct 1 1999 Rename disk c t d column names to disk runp c t d to better reflect the data being recorded and to allow for more per disk information later Version 1 13 Sep 24 1999 Fix a bug in the disk mean calculation where it was being divided by the wrong disk count Now it should be much larger and in scale with disk peak When WATCH DISK is defined now print each disk s run percent Add a new define WATCH MOUNTS which reports each local mount point s disk space and inode capacity usage available for non root users and percent used This comes from Duncan Lawie tyger hoopoes com Add some smarts so that if the number of interfaces physical disks or mounted partitions changes then a new header is printed This will prevent column name and data mixups when the system configuration changes Version 1 12 Sep 14 1999 Add the page scan rate as scanrate in measure cpu Version 1 11 Aug 13 1999 Add the number of CPUs as ncpus Move measure disk and measure ram sooner in the list of subsystems to handle Increase the number of characters for each network interface from four to five Add new disk reads writes Kbytes read and Kbytes written per second Add number of bytes of free memory in bytes as freememK Version 1 10 Jul 28 1999 Measure the process spawn rate if WATCH CPU is defined and the user is root Version 1 9 Jun 2 1999 If WATCH YAHOO is defined then process the access log as a Yahoo style access log Restructure the code to handle different web server access log formats Version 1 8 Jun 1 1999 If the environmental variable WEB SERVER is defined use its value of the as the name of the process to count for the number of web servers on the system If WEB SERVER is not defined then count number of httpd s Version 1 7 Mar 25 1999 Simplify and speed up count proc by 20 Version 1 6 Feb 23 1999 Print pvm user time and system time correctly Version 1 5 Feb 23 1999 Always write header to a new file Version 1 4 Feb 19 1999 Handle missing HTTP 1 x in access log Version 1 3 Feb 18 1999 On busy machines httpops5 will be enlarged Version 1 2 Feb 18 1999 Output data on integral multiples of interval Version 1 1 Feb 18 1999 Integrate Squid log processing from SE 3 1 Version 1 0 Sep 9 1998 Initial version The default sampling interval in seconds define SAMPLE INTERVAL 300 The maximum number of columns of data define MAX COLUMNS 2048 Use the new raw disk code all the time because it has been very well tested and there are no major bug reports in it define USE RAWDISK 1 If WATCH OS is defined then measure every part of the operating system ifdef WATCH OS define WATCH CPU 1 define WATCH MUTEX 1 define WATCH NET 1 define WATCH TCP 1 define WATCH NFS CLIENT 1 define WATCH NFS SERVER 1 define WATCH MOUNTS 1 define WATCH DISK 1 define WATCH DNLC 1 define WATCH INODE 1 define WATCH RAM 1 define WATCH PAGES 1 endif Keep backwards compatibility with WATCH HTTPD ifdef WATCH HTTPD define WATCH WEB 1 endif Keep backwards compatibility with WATCH NFS ifdef WATCH NFS ifndef WATCH NFS CLIENT define WATCH NFS CLIENT 1 endif endif include include include include include include include include include include include include include include include include ifdef WATCH MOUNTS include include endif if WATCH CPU WATCH WEB include ifdef WATCH CPU This is the maximum pid on Solaris hosts define DEFAULT MAXPID 30000 include endif ifdef WATCH WEB Define this macro which returns the size index for a file of a particular size This saves the overhead of a function call define WWW SIZE INDEX size size index if size 1M long dwnld totalz Total size counted from log if WATCH PROXY WATCH SQUID WATCH YAHOO If we re watching a Yahoo log then take the transfer time to be the processing time double www dwnld time sum Transfer time double www dwnld time by size 5 Mean transfer time by size bin endif if WATCH PROXY WATCH SQUID long prxy squid indirect hits that go via PROXY SOCKS parent long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif ifdef USE RAWDISK include This code was developed so that the performance of virtual disks originating from a Sun A1000 raid controller could be monitored These disks do not show up in the GLOBAL disk io structure of SE This extension accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface that matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int first len char first name 16 char long name 16 char short name 16 int i int j This section is used to map short names to long names Since raw disk update has already identified how many physical devices it simply tries to find theses devices in GLOBAL disk info SE appears to have a bug where GLOBAL diskinfo size can be larger than the number of entries in GLOBAL disk info under a variety of conditions In later versions of SE GLOBAL diskinfo size has been removed This appears to fix the above problem This code uses MAX RAWDISKS for the table length and the assumption that short disks names come before short disk partition names to detect the end of the table If it fails to detect the end it will coredump when it addresses unallocated memory strcpy first name GLOBAL disk info 0 short name first len strlen first name for i 0 i 0 if strncmp first name short name first len 0 break if strcmp RAW disk i short name short name 0 strcpy long name GLOBAL disk info j long name strcpy RAW disk i long name long name break RAW disk map 0 raw disk update int rdisk ulong ul kstat ctl t kc 1 kstat t kp 1 kstat t nkp 1 kstat io t kio ulonglong nread ulonglong nwritten uint reads uint writes longlong wtime longlong wlentime longlong wlastupdate longlong rtime longlong rlentime longlong rlastupdate longlong wcnt longlong rcnt double read writes double big etime double elapsed etime double hz etime double nanosecond NANOSEC double update double delta timeval t time update 1 ulong time void char short name 8 gettimeofday time update time void update time update 0 tv sec time update 0 tv usec 1000000 0 delta update RAW disk lastupdate RAW disk lastupdate update kc 0 kstat open Read them if kstat read kc kp 0 1 perror raw disk update kstat read exit 1 Traverse the chain looking for IO events for ul kc 0 kc chain ul 0 ul nkp 0 ks next struct fill nkp 0 ul if nkp 0 ks type KSTAT TYPE IO strcpy short name nkp 0 ks name if short name 0 m short name 0 n strchr short name nil Try to locate device for rdisk 0 rdisk nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong ofile File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column string col previous comment MAX COLUMNS Previous comments int current column 0 The current column int previous number columns 1 Number columns printed last int print header 1 Flag to flush header This is a list of the extensions the compress programs add to the compress filename define NUMBER COMPRESS SUFFIXES 3 string compression suffixes NUMBER COMPRESS SUFFIXES Z gz bz2 Add one column of comments and data to the buffers put output string comment string data if current column MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column Send the stored columns of information to the output print columns string data int i for i 0 i mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in the long interval time span use the difference between the previous and current mpid over a 5 second interval to calculate the long interval difference mpid current mpid5 diff mpid now mpid5 now else sleep sleep till now else sleep sleep till now endif now time 0 ifdef WATCH CPU measure cpu p vmstat pvm double mpid interval double mpid rate pvm vmglobal total In SE 3 0 and below user time and system time are int and in SE 3 1 and above they are double so cast everything to double using 0 0 put output usr sprintf 5 1f pvm user time 0 0 put output sys sprintf 5 1f pvm system time 0 0 put output wio sprintf 5 1f pvm wait time 0 0 put output idle sprintf 5 1f pvm idle time 0 0 put output 1runq sprintf 6 2f tmp kstat misc avenrun 1min 256 0 put output 5runq sprintf 6 2f tmp kstat misc avenrun 5min 256 0 put output 15runq sprintf 6 2f tmp kstat misc avenrun 15min 256 0 put output proc sprintf 5lu tmp kstat misc nproc put output runque sprintf 8 2f pvm runque 0 0 put output waiting sprintf 8 2f pvm waiting 0 0 put output swpque sprintf 8 2f pvm swpque 0 0 put output scanrate sprintf 8 3f pvm scan 0 0 Calculate the rate of new process spawning if can read kernel 0 mpid interval mpid now mpid then 0 000000001 mpid rate mpid current mpid previous mpid interval put output proc s sprintf 7 3f mpid rate put output proc p5s sprintf 9 4f mpid5 rate Reset counters mpid then mpid now mpid previous mpid current mpid5 rate 0 endif ifdef WATCH MUTEX measure mutex put output smtx sprintf 5d tmp lr mutex smtx put output smtx cpu sprintf 8d tmp lr mutex smtx GLOBAL pvm ncpus endif ifdef WATCH NET measure net int previous count 1 int current count int i current count 0 for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads RAW disk i reads total writes RAW disk i writes total readk RAW disk i kreads total writek RAW disk i kwrites mean disk busy RAW disk i run percent if RAW disk i run percent peak disk busy peak disk busy RAW disk i run percent ifdef HAVE EMC DISK CONTROL endif else for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads GLOBAL disk i reads total writes GLOBAL disk i writes total readk GLOBAL disk i kreads total writek GLOBAL disk i kwrites mean disk

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.31.txt (2016-04-24)
    Open archived version from archive


  • to record NFS v2 and v3 server statistics The new statistics are nfss calls the number of NFS calls to the NFS server nfss bad the number of bad NFS calls per second and v 2 3 reads writes which are nfss calls broken down into NFS version 2 and NFS version 3 calls The sum of v 2 3 reads writes will be less than nfss calls as the other types of NFS calls such as getattr and lookup are not included Contributed by Paul Haldane This code is enabled by the standard DWATCH OS or individually by DWATCH NFS SERVER The define DWATCH NFS has been superseded by DWATCH NFS CLIENT but to keep backwards compatibility DWATCH NFS CLIENT will be defined if DWATCH NFS is defined Version 1 21 Jan 12 2000 Prevent core dumps on extremely long access log lines Version 1 20 Oct 20 1999 Update my email address Version 1 19 Oct 13 1999 Prevent a division by zero in calculating the mean disk busy if the number of disks on the system is 0 Version 1 18 Oct 12 1999 Rename disk runp c t d to disk runp c t d to remove the s Version 1 17 Oct 8 1999 Do not record mount point statistics for locally mounted cdrom partitions Version 1 16 Oct 7 1999 To keep backwards compatibility define WATCH WEB if WATCH HTTPD is defined If the COMPRESSOR environmental variable is defined then when a new log file is opened for a new day the just closed log file is compressed using the COMPRESSOR command in the following manner system sprintf s s COMPRESSOR log file COMPRESSOR should be set to something like gzip 9 or compress or bzip2 9 Version 1 15 Oct 5 1999 kvm mpid is a int not a long This caused problems on Solaris 7 hosts running a 64 bit kernel Version 1 14 Oct 1 1999 Rename disk c t d column names to disk runp c t d to better reflect the data being recorded and to allow for more per disk information later Version 1 13 Sep 24 1999 Fix a bug in the disk mean calculation where it was being divided by the wrong disk count Now it should be much larger and in scale with disk peak When WATCH DISK is defined now print each disk s run percent Add a new define WATCH MOUNTS which reports each local mount point s disk space and inode capacity usage available for non root users and percent used This comes from Duncan Lawie tyger hoopoes com Add some smarts so that if the number of interfaces physical disks or mounted partitions changes then a new header is printed This will prevent column name and data mixups when the system configuration changes Version 1 12 Sep 14 1999 Add the page scan rate as scanrate in measure cpu Version 1 11 Aug 13 1999 Add the number of CPUs as ncpus Move measure disk and measure ram sooner in the list of subsystems to handle Increase the number of characters for each network interface from four to five Add new disk reads writes Kbytes read and Kbytes written per second Add number of bytes of free memory in bytes as freememK Version 1 10 Jul 28 1999 Measure the process spawn rate if WATCH CPU is defined and the user is root Version 1 9 Jun 2 1999 If WATCH YAHOO is defined then process the access log as a Yahoo style access log Restructure the code to handle different web server access log formats Version 1 8 Jun 1 1999 If the environmental variable WEB SERVER is defined use its value of the as the name of the process to count for the number of web servers on the system If WEB SERVER is not defined then count number of httpd s Version 1 7 Mar 25 1999 Simplify and speed up count proc by 20 Version 1 6 Feb 23 1999 Print pvm user time and system time correctly Version 1 5 Feb 23 1999 Always write header to a new file Version 1 4 Feb 19 1999 Handle missing HTTP 1 x in access log Version 1 3 Feb 18 1999 On busy machines httpops5 will be enlarged Version 1 2 Feb 18 1999 Output data on integral multiples of interval Version 1 1 Feb 18 1999 Integrate Squid log processing from SE 3 1 Version 1 0 Sep 9 1998 Initial version The default sampling interval in seconds define SAMPLE INTERVAL 300 The maximum number of columns of data define MAX COLUMNS 2048 Use the new raw disk code all the time because it has been very well tested and there are no major bug reports in it define USE RAWDISK 1 If WATCH OS is defined then measure every part of the operating system ifdef WATCH OS define WATCH CPU 1 define WATCH MUTEX 1 define WATCH NET 1 define WATCH TCP 1 define WATCH NFS CLIENT 1 define WATCH NFS SERVER 1 define WATCH MOUNTS 1 define WATCH DISK 1 define WATCH DNLC 1 define WATCH INODE 1 define WATCH RAM 1 define WATCH PAGES 1 endif Keep backwards compatibility with WATCH HTTPD ifdef WATCH HTTPD define WATCH WEB 1 endif Keep backwards compatibility with WATCH NFS ifdef WATCH NFS ifndef WATCH NFS CLIENT define WATCH NFS CLIENT 1 endif endif include include include include include include include include include include include include include include include include ifdef WATCH MOUNTS include include endif if WATCH CPU WATCH WEB include ifdef WATCH CPU This is the maximum pid on Solaris hosts define DEFAULT MAXPID 30000 include endif ifdef WATCH WEB Define this macro which returns the size index for a file of a particular size This saves the overhead of a function call define WWW SIZE INDEX size size index if size 1M long dwnld totalz Total size counted from log if WATCH PROXY WATCH SQUID WATCH YAHOO If we re watching a Yahoo log then take the transfer time to be the processing time double www dwnld time sum Transfer time double www dwnld time by size 5 Mean transfer time by size bin endif if WATCH PROXY WATCH SQUID long prxy squid indirect hits that go via PROXY SOCKS parent long prxy squid cache hits hits returned from cache endif ifdef WATCH PROXY long prxy cache writes Number of writes and updates to cache long prxy uncacheable Number of explicitly uncacheable httpops Any extra is errors or incomplete ops endif ifdef WATCH SQUID long squid cache misses long squid icp requests long squid icp queries long squid client http endif endif ifdef USE RAWDISK include This code was developed so that the performance of virtual disks originating from a Sun A1000 raid controller could be monitored These disks do not show up in the GLOBAL disk io structure of SE This extension accesses the sys kstat se interface to the kstat IO queues to extract info on drives not available in the kstat se kstat disk interface Global data shared between function calls struct RawDisk Exposed interface that matches kstat char long name 16 char short name 8 double reads double kreads double writes double kwrites double avg wait double avg serv double service double wait percent double run percent Hidden internal registers to track sys kstats counters int number kstat disk ulonglong nread Number of bytes read ulonglong nwritten Number of bytes written uint reads Number of read operations uint writes Number of write operations longlong wtime Cumulative wait pre service time longlong wlentime Cumulative wait length time product longlong wlastupdate Last time wait queue changed longlong rtime Cumulative run service time longlong rlentime Cumulative run length time product longlong rlastupdate Last time run queue changed uint wcnt Count of elements in wait state uint rcnt Count of elements in run state Define global for tracking raw disk data define MAX RAWDISKS 512 RawDisk RAW disk MAX RAWDISKS int RAW disk map 0 int RAW disk count 0 double RAW disk lastupdate Function to scan kstat and map short device names to long device names raw disk map int first len char first name 16 char long name 16 char short name 16 int i int j This section is used to map short names to long names Since raw disk update has already identified how many physical devices it simply tries to find theses devices in GLOBAL disk info SE appears to have a bug where GLOBAL diskinfo size can be larger than the number of entries in GLOBAL disk info under a variety of conditions In later versions of SE GLOBAL diskinfo size has been removed This appears to fix the above problem This code uses MAX RAWDISKS for the table length and the assumption that short disks names come before short disk partition names to detect the end of the table If it fails to detect the end it will coredump when it addresses unallocated memory strcpy first name GLOBAL disk info 0 short name first len strlen first name for i 0 i 0 if strncmp first name short name first len 0 break if strcmp RAW disk i short name short name 0 strcpy long name GLOBAL disk info j long name strcpy RAW disk i long name long name break RAW disk map 0 raw disk update int rdisk ulong ul kstat ctl t kc 1 kstat t kp 1 kstat t nkp 1 kstat io t kio ulonglong nread ulonglong nwritten uint reads uint writes longlong wtime longlong wlentime longlong wlastupdate longlong rtime longlong rlentime longlong rlastupdate longlong wcnt longlong rcnt double read writes double big etime double elapsed etime double hz etime double nanosecond NANOSEC double update double delta timeval t time update 1 ulong time void char short name 8 gettimeofday time update time void update time update 0 tv sec time update 0 tv usec 1000000 0 delta update RAW disk lastupdate RAW disk lastupdate update kc 0 kstat open Read them if kstat read kc kp 0 1 perror raw disk update kstat read exit 1 Traverse the chain looking for IO events for ul kc 0 kc chain ul 0 ul nkp 0 ks next struct fill nkp 0 ul if nkp 0 ks type KSTAT TYPE IO strcpy short name nkp 0 ks name if short name 0 m short name 0 n strchr short name nil Try to locate device for rdisk 0 rdisk nread nread RAW disk rdisk nread reads kio reads if RAW disk rdisk reads reads reads RAW disk rdisk reads rlentime kio rlentime rtime kio rtime rlastupdate kio wlastupdate rcnt kio rcnt nwritten kio nwritten if RAW disk rdisk nwritten nwritten nwritten RAW disk rdisk nwritten writes kio writes if RAW disk rdisk writes writes writes RAW disk rdisk nwritten wlentime kio wlentime wtime kio wtime wlastupdate kio wlastupdate wcnt kio wcnt elapsed etime wlastupdate RAW disk rdisk wlastupdate if elapsed etime 0 hz etime elapsed etime nanosecond big etime 1024 0 hz etime else elapsed etime nanosecond hz etime 1 0 big etime 1024 0 RAW disk rdisk reads reads RAW disk rdisk reads hz etime RAW disk rdisk kreads nread RAW disk rdisk nread big etime RAW disk rdisk writes writes RAW disk rdisk writes hz etime RAW disk rdisk kwrites nwritten RAW disk rdisk nwritten big etime read writes elapsed etime RAW disk rdisk reads RAW disk rdisk writes 1024 0 if read writes 0 RAW disk rdisk avg wait wlentime RAW disk rdisk wlentime read writes RAW disk rdisk avg serv rlentime RAW disk rdisk rlentime read writes RAW disk rdisk service RAW disk rdisk avg wait RAW disk rdisk avg serv else RAW disk rdisk avg wait 0 0 RAW disk rdisk avg serv 0 0 RAW disk rdisk service 0 0 Update the counters RAW disk rdisk run percent 100 0 rtime RAW disk rdisk rtime elapsed etime RAW disk rdisk wait percent 100 0 wtime RAW disk rdisk wtime elapsed etime RAW disk rdisk writes writes RAW disk rdisk nwritten nwritten RAW disk rdisk wlastupdate wlastupdate RAW disk rdisk wlentime wlentime RAW disk rdisk wtime wtime RAW disk rdisk wcnt wcnt RAW disk rdisk reads reads RAW disk rdisk nread nread RAW disk rdisk rlastupdate rlastupdate RAW disk rdisk rlentime rlentime RAW disk rdisk rtime rtime RAW disk rdisk rcnt rcnt kstat close kc Map long device names for any drives that we just discovered if RAW disk map 1 raw disk map endif RAWDISK Variables for handling output string compress getenv COMPRESSOR How to compress logs ulong out log fp File pointer to the logging file string col comment MAX COLUMNS Comments for each column string col data MAX COLUMNS Data for each column string col previous comment MAX COLUMNS Previous comments int current column 0 The current column int previous number columns 1 Number columns printed last int print header 1 Flag to flush header This is a list of the extensions the compress programs add to the compress filename define NUMBER COMPRESS SUFFIXES 3 string compression suffixes NUMBER COMPRESS SUFFIXES Z gz bz2 Add one column of comments and data to the buffers put output string comment string data if current column MAX COLUMNS fprintf stderr s too many columns d Increase MAX COLUMNS n program name current column exit 1 col comment current column comment col data current column data current column Send the stored columns of information to the output print columns string data int i for i 0 i mpid5 previous mpid5 diff mpid5 current mpid5 previous else mpid5 diff mpid5 current DEFAULT MAXPID mpid5 previous rate mpid5 diff mpid5 interval if rate mpid5 rate mpid5 rate rate mpid5 previous mpid5 current Now take these results to measure the long interval rate Because the mpid may flip over DEFAULT MAXPID more than once in the long interval time span use the difference between the previous and current mpid over a 5 second interval to calculate the long interval difference mpid current mpid5 diff mpid now mpid5 now else sleep sleep till now else sleep sleep till now endif now time 0 ifdef WATCH CPU measure cpu p vmstat pvm double mpid interval double mpid rate pvm vmglobal total In SE 3 0 and below user time and system time are int and in SE 3 1 and above they are double so cast everything to double using 0 0 put output usr sprintf 5 1f pvm user time 0 0 put output sys sprintf 5 1f pvm system time 0 0 put output wio sprintf 5 1f pvm wait time 0 0 put output idle sprintf 5 1f pvm idle time 0 0 put output 1runq sprintf 6 2f tmp kstat misc avenrun 1min 256 0 put output 5runq sprintf 6 2f tmp kstat misc avenrun 5min 256 0 put output 15runq sprintf 6 2f tmp kstat misc avenrun 15min 256 0 put output proc sprintf 5lu tmp kstat misc nproc put output runque sprintf 8 2f pvm runque 0 0 put output waiting sprintf 8 2f pvm waiting 0 0 put output swpque sprintf 8 2f pvm swpque 0 0 put output scanrate sprintf 8 3f pvm scan 0 0 Calculate the rate of new process spawning if can read kernel 0 mpid interval mpid now mpid then 0 000000001 mpid rate mpid current mpid previous mpid interval put output proc s sprintf 7 3f mpid rate put output proc p5s sprintf 9 4f mpid5 rate Reset counters mpid then mpid now mpid previous mpid current mpid5 rate 0 endif ifdef WATCH MUTEX measure mutex put output smtx sprintf 5d tmp lr mutex smtx put output smtx cpu sprintf 8d tmp lr mutex smtx GLOBAL pvm ncpus endif ifdef WATCH NET measure net int previous count 1 int current count int i current count 0 for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL if pioGLOB old wlentime i pioGLOB old rlentime i 1 endif total reads RAW disk i reads total writes RAW disk i writes total readk RAW disk i kreads total writek RAW disk i kwrites mean disk busy RAW disk i run percent if RAW disk i run percent peak disk busy peak disk busy RAW disk i run percent ifdef HAVE EMC DISK CONTROL endif else for i 0 i Check wr lentime to see if an EMC is using a fake disk for control EMC disks have a fake disk which commands are run over to configure the disk array or to get stats from they are not real data transfers They cause 1000 MB sec writes to appear in the stats I still get them but not as often with this bit of code in If the I O which occurred in the last five minutes is not greater than 1 100sec then it is not a valid measurement anyway What happens is that we can have a small I O say 1024 bytes in a 1 100sec 1024 100 sec I am thinking of making it wlentime rlentime 2 since I am still getting fake write spikes ifdef HAVE EMC DISK CONTROL

    Original URL path: https://www.orcaware.com/orca/pub/orcallator.se-1.32.txt (2016-04-24)
    Open archived version from archive



  •