User talk:DogNut/Archive01
From WWR
Hey DogNut! You're the 1st person to sign up here ... so I made you a sysop. Hehehe...just thought if you're playing that having sysop power would be worthwhile. Not sure how (or if) this wiki will evolve, but hopefully folks with a lot of time on their hands who like to document stuff will jump right in. I'm guessing that's neither you nor me. :-)
Contents |
Old Site stuff
Sparkit, I noticed that you started the "Old Site" template to the explaintion of pages in the old site. I am wondering if that means spending a lot of time detailing things out like the Advanced Console is probably not worth it right now? I just want make sure I am spending my time in the right places. Thanks DogNut 03:44, 22 March 2006 (AKST)
- Hi DogNut! Thanks for asking. That is exactly why I'm flagging them. Jimbob has cautioned about documenting the old site. Perhaps becuse he expects to have it all in her soon. My sense is that we'll have all or most of the features, but the interface will be different. I'm also hoping to alleviate some confusion by distinguishing between version 2 (old) and version 4 (new). I wouldn't spend too much time on the Advanced Console. Sparkit 03:55, 22 March 2006 (AKST)
New: page deletion requests page
We now have a Page deletion requests page for users to request deletion of pages, and admins to watch for deletion requests. We'll probably get most of the requests in the chats, but it's an alternative. Also an Category:Admin for that sort of bureaucratic BS.
[[Image:bigbday.gif Happy Day, DogNut and happy year....while i'm at it happy life lol ReWoven (talk) 11:06, May 1, 2008 (AKDT)
Artists pages
- Sparkit, what's the deal with (Red > Blue)? I tried picking an red artist and saving them, is that what you are doing except you are adding a comment (Red > Blue)? Am I missing something, that seem's too simple to be correct. DogNut 17:33, September 7, 2006 (AKDT)
- Hiya, DogNut! All I do is hit the red link from a link that has the artists extension at the end (most of the dynamic pages - that extension adds the boilerplate stuff into the edit box). Add the "red link > blue link" summary comment. Click "Save".
- I also set my edit pref to have "This is a minor edit" checked by default. Makes it easy to filter these out in the change list.
- Yup, it's that simple. (Whasername 17:39, September 7, 2006 (AKDT))
- You are most welcome. I did neglect to mention that I add cross-references (See also) when I'm aware of them. Looks like you've already picked that up.
bigblink
- You are most welcome. I did neglect to mention that I add cross-references (See also) when I'm aware of them. Looks like you've already picked that up.
- In some cases when more than one of the WWR links shows ... ugh! I'm having a hard time explaining this ... Calvin Owens shows the Calvin Owens And Trudy Lynn, and there aren't any other Trudy Lynn tracks, I make the one with the fewer tracks a redirect.
- I think I've given up cross-referencing Esther Golton though. Enjoy! (Sparkit 18:58, September 7, 2006 (AKDT))
- Gotcha! As usual, I just need to slow down and look around a little. Please don't be bashful if I am screwing things up! Thanks, DogNut 19:07, September 7, 2006 (AKDT)
- Screw things up?!
hund19 I just make this stuff up as I go along, and expect other folks to do the same. It's damn near impossible to screw things up anyhow. I've just found a few tricks with making artists pages is all. (Sparkit 19:20, September 7, 2006 (AKDT))
- Screw things up?!
Paddy Gibney
Spartit, can you take a look at Paddy Gibney and Paddy Gibney and Bill Ayerbe. I am thinking I should redirect Paddy Gibney and Bill Ayerbe at Paddy Gibney. What do you think? Thanks DogNut 15:28, September 9, 2006 (AKDT)
- That's what I'd do... Redirect Paddy Gibney and Bill Ayerbe to Paddy Gibney, because if you look at the Paddy Gibney page in the auto-generated artist box the Paddy Gibney and Bill Ayerbe album One Drunken Night is listed which means all that album's tracks are available on that page, plus a search of the music library turns up no other listings for Bill Ayerbe. I know that temporarily nixes a biography for Bill Ayerbe, but redirects are easy to undo if someone ever decides Bill Ayerbe needs a bio page. (Sparkit 15:50, September 9, 2006 (AKDT))
Quick Note
Quick note: Googlebot just hit. Am giving this layout stuff now so you can interpret any logs I send. Leave me a note if any questions.
Google hit:
06/09/09 15:53:55|66.249.66.234|"-"|"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"|"GET /robots.txt HTTP/1.1"|200|0|+|2086|247|F-3
Log Layout: %{%y/%m/%d %H:%M:%S}t|%h|\"%{Referer}i\"|\"%{User-agent}i\"|\"%r\"|%>s|%T|%X|%O|%I|F-3
Fields description in case you don't have:
The following directives define some format nicknames for use with a CustomLog directive (see below).
%...a: Remote IP-address
%...A: Local IP-address
%...B: Bytes sent, excluding HTTP headers.
%...b: Bytes sent, excluding HTTP headers. In CLF format
. i.e. a '-' rather than a 0 when no bytes are sent.
%...c: %X is the alias?? Connection status when response was completed.
. 'X' = connection aborted before the response completed.
. '+' = connection may be kept alive after the response is sent.
. '-' = connection will be closed after the response is sent.
%...{FOOBAR}e: The contents of the environment variable FOOBAR
%...f: Filename
%...h: Remote host
%...H The request protocol
%...{Foobar}i: The contents of Foobar: header line(s) in the request
. sent to the server.
%...l: Remote logname (from identd, if supplied)
%...m The request method
%...{Foobar}n: The contents of note "Foobar" from another module.
%...{Foobar}o: The contents of Foobar: header line(s) in the reply.
%...p: The canonical Port of the server serving the request
%...P: The process ID of the child that serviced the request.
%...q The query string (prepended with a ? if a query string exists,
. otherwise an empty string)
%...r: First line of request
%...s: Status. For requests that got internally redirected, this is
. the status of the *original* request --- %...>s for the last.
%...t: Time, in common log format time format (standard english format)
%...{format}t: The time, in the form given by format, which should
. be in strftime(3) format. (potentially localized)
%...T: The time taken to serve the request, in seconds.
%...u: Remote user (from auth; may be bogus if return status (%s) is 401)
%...U: The URL path requested, not including any query string.
%...v: The canonical ServerName of the server serving the request.
%...V: The server name according to the UseCanonicalName setting.
F-3 is just my tag for my own version # so I know what format the log is in
You need to enable mod_logio.c to use %I and %O %I=actual input bytes %O=actual output bytes
Entire Log:
06/09/09 15:47:15|216.152.181.194|"-"|"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6"|"GET /robots.txt HTTP/1.1"|200|0|+|2086|601|F-3
06/09/09 15:48:41|195.250.198.65|"-"|"Mozilla/5.0 (compatible; TridentSpider/3.1)"|"GET /robots.txt HTTP/1.1"|200|0|+|2086|294|F-3
06/09/09 15:49:19|66.94.237.181|"-"|"YahooFeedSeeker Testing/2.0 (compatible; Mozilla 4.0; MSIE 5.5; http://publisher.yahoo.com/rssguide)"|"GET /robots.txt HTTP/1.0"|304|0|+|196|377|F-3
06/09/09 15:50:44|72.30.252.167|"-"|"Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"|"GET /robots.txt HTTP/1.0"|200|0|-|2049|200|F-3
06/09/09 15:53:55|66.249.66.234|"-"|"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"|"GET /robots.txt HTTP/1.1"|200|0|+|2086|247|F-3
06/09/09 15:54:19|66.218.65.27|"-"|"YahooFeedSeeker/2.0 (compatible; Mozilla 4.0; MSIE 5.5; http://publisher.yahoo.com/rssguide)"|"GET /robots.txt HTTP/1.0"|200|0|+|2086|313|F-3
06/09/09 15:54:38|66.94.237.181|"-"|"YahooFeedSeeker Testing/2.0 (compatible; Mozilla 4.0; MSIE 5.5; http://publisher.yahoo.com/rssguide)"|"GET /robots.txt HTTP/1.0"|304|0|+|196|377|F-3
06/09/09 15:57:04|209.237.238.179|"-"|"ia_archiver"|"GET /robots.txt HTTP/1.0"|200|0|-|2049|128|F-3
06/09/09 15:58:37|72.30.129.35|"-"|"Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"|"GET /robots.txt HTTP/1.0"|200|0|-|2049|200|F-3
06/09/09 15:59:01|66.94.237.181|"-"|"YahooFeedSeeker Testing/2.0 (compatible; Mozilla 4.0; MSIE 5.5; http://publisher.yahoo.com/rssguide)"|"GET /robots.txt HTTP/1.0"|304|0|+|196|377|F-3
Server
Ken ...
This morning (Friday, Oct 13 2006 about 9:20-9:30) load went to 35+ ... I saw a lot of apache processes running and using my alias "doit" (try it if you want) saw a particular IP in Europe was trying to load 50+ pages. Don't know if it was a hacker or bot but added MaxClient 30 to Apache conf file. Apparently it will allow more than 30 concurrent processes, but queues any above 30 instead of executing immediately. Thanks for giving me a direction to go. We'll watch it.
Oh - and I see 2 users logged onto server. Linux command to send text screen message to other user (assumed to be you) or see their IP so I can confirm it's you????
Jimbob 09:57, October 13, 2006 (AKDT)
Oh, and I just had to reboot server (got nervous about the 2nd root user and wanted to kick him - I think it was a previous logon of me that didn't get shutdown correctly but wasn't sure). Not sure if your cron fires off automatically, but if you're wondering what happened to it, that's why it's dead. One of these days, I'll pass along my cheatsheet, etc. via email so if you need to see how woefully inadequate I am as a Linux admin, you'll know. :-) Jimbob 10:22, October 13, 2006 (AKDT)
Thanks on phone offer. Yeah, send it in case of emergency. Since I can't concentrate on monitoring system I can't really do the job justice of tracking stuff and reading about / tweaking all the Apache config parameters. People just walked in - got to go.... Jimbob 10:26, October 13, 2006 (AKDT)
Jim, I saw you increased the buffers for mysql. I have my capture job running counting httpd procs, doing a iostat -dx hda3, and a vmstat every 30 seconds so we can see how things balance out. The log is in the usual place. You can also run sar, a couple of examples are "sar -rq | more" which would show memory utilization and CPU load since the beginning of the day. If you want to add a start time, you can "sar -rq -s 07:00:00 | more" (note you need the seconds to make it work). You can also add a "d" to the -rq and get disk statistics, but that shows up the bug I talked about in my notes so don't look at avgqu-sz svctm or %util columns. DogNut 07:26, October 22, 2006 (AKDT)
- You're really good. I was gonna drop you an email about the mysql buffer changes but it got late. I've spent enough time in your directory to learn a little bit about iostat and vmstat but I'm not sophisticated enough to really have a handle on how changing buffers etc. effects overall system performance ... but it seems a tad less bogged down.
smile Glad you're keeping an eye on it. The big thing was that I had never uncommented the innodb buffer sizes when I went to the wiki - which uses innodb tables heavily.
- The high ratio of writes to reads on disk bugs me for 2 reasons: 1) writes are usually slower than reads but more importantly 2) who the hell is writing so much? If I'm reading iostat -dx hda3 correctly, it's not unusual for there to be 20x more writes than reads. I was tempted to reboot the machine just to start all those stats at 0 to see if the innodb buffers were helping, but if I'm reading iostat -dx hda3 5 100 correctly, I'm seeing the ratio of reads to writes in 5 second time slices anyway. I googled as best I could to find if it was a common Linux problem but didn't get anywhere. It has to be the wiki software / the way innodb tables are being handled (mySQL stats pages seem to indicate a lot of innodb writing) or else something buried in the system cuz I sure don't think my apps are writing that much. It's a good mystery - if there is a util that can snapshot the names of the file handles being written to, we might be able to see who's writing so much???
- The way the box was setup (by 1&1) was with Plex(? Plesk?) ... they expect people to farm out the server to their own customers. I did my best to disable it at bootup time but there may still be unneeded processes associated with it. When I do a ps -ef I see some stuff that I don't know what it is or why it's loaded or how essential it is. Most of it doesn't seem to be taking much CPU (except the double entry for the journaling system?) so I haven't worried about it. Journal and mySQL seem to be the CPU hogs - especially if the httpd web-based stats page CPU utilization which is normally under 10% is to be believed. And, if the CPU idle numbers are correct, the box may be bottlenecked at all the disk I/O and not at the CPU. If you see some processes that are unneeded, don't hesitate to kill them / remove them from the bootup file. Perhaps one of them is unhappy that another process it expects to be running is no longer alive and is writing error logs out the wazoo???
- It may seem pointless to spend so much time tuning a server which we'll likely migrate off of in the coming months, but every little thing I can learn about Linux is both useful and fun. And I can't tell you how wonderful it's been for you to take charge and show me what utils are useful and how to run them ... Linux has so many utils that I could never figure out which ones everyone else used to tune their system. If/when we go to a new box, my druthers would be to more or less wipe it clean, then have you do/teach me how to install Linux and use the RFP(?? whatever the system is called to keep the core updated). I'd like to install a play Linux system here at home but need to rein myself in and prioritize enhancing the PHP apps and not getting too buried in learning the bowels of Linux. (For example, I still haven't used any of the wiki-supplied utils to purge unused photos, dead wiki pages, etc. and I notice we keep eating up more disk for user-uploaded 1-time posting photos etc....I'll have to get on that in the coming months.)
- One more question ... how did you find that I'd changed the buffers in my.conf so quickly? Were you monitoring that file itself (httpd.conf and my.conf and the bootup, sudo, and alias files are the only system files I've messed with I think and it took me forever to find them and get up the guts to modify them) or do you have some log you look at to see which files have been recently modified? BTW, it's also been great working with an expert like you cuz I know you've got an eye on security so if I publicly mention something I shouldn't have, you'll let me know... Jimbob 10:19, October 22, 2006 (AKDT)
Hits from 127.0.0.1
xxxxxx... 127.0.0.1 www.wholewheatradio.org GET / HTTP/1.0
I can't stand it ... on the server-status, any idea what are all the hits from 127.0.0.1 that appear to be loading the top-level page? I can't figure that out but figure it's some internal processing ... are they are child processes of the original request (they never have a PID)??? ... any clues? Jimbob 12:36, October 22, 2006 (AKDT)
- The only thing I have been able to figure out is that 127.0.0.1 shows up when there is no PID associated with that slot. Httpd must keep a table the size of MaxClients that it uses to keep track of the processes. Anyway, my guess is it just put's the localhost address in there to show no one is connected. I saw your post about Esther, I hope it works out for her. I guess you get a reprieve from Painting??? Anyway, I need to run out and pick up Ryan, so I'll catch you in a few. DogNut 12:51, October 22, 2006 (AKDT)
- Got it. Yeah, probably an empty slot. From google "...You should see 20 apache children. Once the children become idle, one should move from a "." (idle) to a "_" (no process) each second." This from the blind-leading-the-blind but since they have them too and don't seem to care I won't either. Jimbob 13:03, October 22, 2006 (AKDT)
Googlebot
Found this guy's blog - lots of comments about googlebot: http://www.mattcutts.com/blog/more-webmaster-console-goodness/ Just interesting to see other people's complaints etc. If I followed it for awhile (and cared enough) I might leave a comment about specific cases where googlebot just ignores robots.txt ... but I don't think I want to get that involved in making 100% sure I've got it right. If you do, have at it.
I changed the crawl rate using Goggle's webmaster tools page to "slow". We'll see if it does slow down. If you look at the page load stats, it's amazing. The average WWR pages crawled by googlebot over the last 90 days is 12,483 ... here are the stats:
Maximum Average Minimum
Number of pages crawled per day: 25174 12483 266
Number of kilobytes downloaded per day: 119545 54700 114
Time spent downloading a page (in milliseconds): 1937 1104 17
Jimbob 00:26, October 23, 2006 (AKDT)
FYI System Junk
I changed php.ini (usr/local/xxxyou'll see it) to give 15 seconds before canceling a PHP page (some slower loading ones were crapping out which you would see in the php error log) and also changed some memory parms - comments with current date are the ones I changed. So if you see an overall memory change, that's what did it. System seems faster right now - could be those changes as well as a few internal buffering changes I made in the wiki software config file (Localsettings.xxx in wiki dir). Was just goofing around and ended up going until 4:30. Not in programmer's head ... just testing out the waters and trying to get used to doing it without smoke. Jimbob 04:45, November 7, 2006 (AKST)
LizLytle
Jim,
I saw you uploading the Lizlytle pics. Is she your older sisters daughter? I took a look at her site and she has a very cool acting background, you must be proud to have another artist in the family! DogNut 17:56, November 30, 2006 (AKST)
- Good eye. I was getting the materials together for this blog entry on Liz. Thanks for the kind words! Jimbob 17:58, November 30, 2006 (AKST)
mySQL
Yup, set it dynmically. query_cache_size to 50000 (I think) ... I tried 40000 but I think it failed as it needs at least 40000. http://www.mysql.com/news-and-events/newsletter/2003-01/a0000000108.html
Variables:
query alloc block size 8192 8192
query cache limit 1048576 1048576
query cache min res unit 4096 4096
query cache size 41984 41984
query cache type ON ON
query cache wlock invalidate OFF OFF
query prealloc size 8192 8192
Stats:
Qcache free blocks 1
Qcache free memory 4528
Qcache hits 74831
Qcache inserts 3247278
Qcache lowmem prunes 3189169
Qcache not cached 787302
Qcache queries in cache 16
Qcache total blocks 39
I don't fully understand those stats: is hits=74,831 the number of hits that were actually in cache so didn't have to be looked up? 'not cached' 787,302 seems self-explanatory. If I'm reading it correctly, then about 10% of hits WERE in the cache. Seems like it should be higher. Whatever.
BTW, I think I just got squid running here locally. Can't tell yet statistically what it's doing (finally got a monitoring program to run) but it does seem to be serving pages faster. All these are local tests now that I have a debug machine. Once I'm a little farther on, I'll email you the address etc. so you can logon too and play if you want. Jimbob 13:32, January 5, 2007 (AKST)
Happy Birthday
Happy Birthday Dognut! I certainly hope you have a great day and I wanted to tell you how much you efforts here at WWR are appreciated! Thank you so much! atuuschaaw 04:20, May 1, 2007 (AKDT)
All the best to you, DogNut! You will have a blast in AK this summer! Hope the year is wonderful for you; filled with laughter and adventure!
Fine-line 10:45, May 1, 2007 (AKDT)
Happy Birthday!
It's strange but ever since I was a kid, I've remembered (and forgotten and then re-remembered) that May 1st is your birthday. I don't know why, but I guess cuz its May Day and the dividing line between the dark, cold winter and the upcoming sunny spring and summer. Your quiet involvement in the behind-the-scenes system stuff was sort of like a May Day birthday for me. (Or is it Mayday Mayday!!??)
Jim 06:38, May 1, 2007 (AKDT)
Jac (talk) 03:05, May 1, 2008 (AKDT)
Yadhtrib Yppah
Ot uoy tungod
Thank you
There have been probably over a thousand times that we should have Thanked You, but if not for the "Crash of '07" and Jim annoucing your hidden involvement of keeping WWR up and up todate, we would have not known. So with us duly noted of your behind the scene activity, we now Thank You for what you have done and because we are forgetful, Thank You for what you will do for us the next time, and the next, and the ...
Photos in the neighborhood
Didn't do any editing, just posted them all as taken. Man, it seemed so much bigger when we were kids! http://picasaweb.google.com/jimkloss01/WhereIGrewUp Feel free to delete the following embedded slideshow if you don't want it on your talk page. Jim 05:12, October 20, 2007 (AKDT)
Congratulations!
We want photos of the new puppy! Jimbob 11:48, November 2, 2007 (AKDT)
