Curt Schilling Too Big For His Breeches!

What a hack! First of all, know that this post is about a Red Sox pitcher by a Yankees fan. Take it as you will.

Mr. Schilling is on injured reserve and the big to-do in our town is that he was pitching tonight for his farm team - the Charlotte Knights. The team, by the way, is actually not even in Charlotte or even in North Carolina for that matter! It's in Fort Mill, SC, the town between Rock Hill, SC and Charlotte, NC.

Anyway, I asked my buddy Chris Moody to get me an autograph on my Yankees baseball cap. I figured Schilling would like that. =) Anyway, the guy wasn't even signing autographs. The dude gets demoted to the farm squad in small-town America and he can't even sign some autographs. My old roomie even drove a whole hour to see him!

Check out Moody's blog for some pics...

Slip the LIC

You iSeries guys know what this is about... Was doing an IPL to load PTFs (CUME, Hipers, DB, etc...) on my backup iSeries 820 today when I got an SRC B6000901. I looked this up and the book said I would have to reload the Licensed Internal Code. I didn't like the sounds of that, but after speaking with support I'm starting that process now.

FYI, we generally don't apply the PTFs permanently before we load new fixes. IBM told me that we should start doing that. This will verify the link loader (which is what our specific issue is), clean up orphaned objects, verify indexes on products, and gain a small amount of disk space back.

I'm coming up to load LIC from the CD now and then loading all the fixes.

Sametime 6.5.1 for iSeries is OK

So I'm told...

I love the Lotus Support Services RSS Feeds. I got a "Flash" hit today that appears to be related to my prior Sametime for iSeries problems I wrote about here and here.

The "Flash" regarding Sametime is here. The workaround is to (a) Remove awareness functionality for Names and (b) to add the following line to each Client notes.ini: IM_SHOW_STATUS=0. After I sent the Lotus Support tech the question if this is our problem, I was told that it appears to be. Now that 90+% of my Notes Clients are at 6.5.2 or higher because Lotus wants us to use integrated IM in the Client, I see that this can actually cause the server to hang. This is wonderful! Below is the text of the Flash:

Abstract
Interaction between Notes client 6.5.2 and higher instant messaging (IM) and the IBM Lotus Instant Messaging and Web Conferencing (Sametime) server causes the CPU on both the Sametime server (STMux and STCommunity processes) and the Notes client (nlnotes process) to spike.

Symptoms of this problem include:

--The nlnotes process on the Notes client spikes to 100%.
--Stmux and StCommuntity processes on the Sametime server are pegged at above normal rates that will eventually reach 100%.
--Logging off from the Notes client instant messaging returns the CPU on the Nlnotes process to normal.

Content
The technote titled "Integrated Awareness on Notes 6.5.2 and Higher Will Result in Server Outages" (#1206369), describes the symptoms, workaround, and fix information for this issue.

Can we bring back the dancing girls?

Well, not sure if I'll get debriefed about this, but it was on the public Lotus FTP site. It's a video titled "storyofnotes.wmv". I found it when looking for the document Ed posted up today. I decided to go up one level and see what else was out there:

ftp://ftp.lotus.com/pub/lotusweb/storyofnotes.wmv

About the Blog title, they have a clip from the Notes R4 release at Lotusphere in 96 with some dancing girls (and guys) on stage - we need that for ND7 in 06!

Anyway, I love the quote when they are showing the video of the Russians with a Lotus Notes Intranet:

"We're Marxists but we gotta make a buck like everybody else."

Another quote:

"We enter the new millenium with 56 million Lotus Notes users"

Interesting that the number has doubled in only 5 years - awesome!

"and it's only the beginning!"

Domino 6.5.4 for iSeries Issue

As you know, I've been having some issues with my mail server hanging after going to 6.5.4. After loading iSeries PTFs and finally getting some debug logs sent to Lotus (it's been about 2 months dealing with this on about a weekly basis), they told me Friday:

"I used several of my resources at IBM to conclude that this appears to be a network problem."

So, I have a communications trace going right now and am collecting random NETSTAT *CNN logs. They also think I should break out a sniffer.

Will keep you updated...

Academic Research?

Did anyone else get "selected" for an Academic Research with Rensselaer Polytechnic Institute (RPI) in Troy, NY?

I got the following e-mail this morning:

Dear Chris Whisonant:

I am a doctoral student at the School of Humanities and Social Sciences, Rensselaer Polytechnic Institute (RPI) in Troy, NY.

This note is to request your participation in an academic research study that I am conducting (with Prof. James Watt of RPI) on online customer/product communities. The goal of this research study is to understand customer experiences in online product communities. Please note that this is an academic study conducted as part of my PhD dissertation work at RPI and is not sponsored by (or affiliated with) any other organization.

As a regular participant in the IBM online customer community, your contribution to this research study will be much appreciated and valued. I would like to invite you to participate in this research by completing the study questionnaire (which will take only 10-12 mminutes) available at the following web site hosted by the Social and Behavioral Research Lab at RPI:

URL

I understand you will have other time commitments. However, I hope you will be able to spare some time to participate and contribute to this research. Thank you much for your time.

I e-mailed him back asking where he got my address and he stated:

I got your email address from the IBM online forum.

I'm assuming he means Notes.net (or LDD or devWorks...) =) Off to take the quiz - what the heck!

Clay Bar for GTP!

Sadly the pictures didn't turn out good of the results... But I decided to use a clay bar on my '01 Pontiac Grand Prix GTP (LOVE that supercharger!!).

I was having a problem with getting lots of tiny black "dust" particles all over parts of my car.

Basically, I'm getting most of it on the very rear of the car - from bumper up to spoiler. There is also some that gets into the rear door cracks. I've had this car for almost a year now and I've seen it since the first time I washed it. It's got to be an environmental problem.

Well, I took the plunge and got a Mother's Clay Bar kit. I was extremely impressed with the results. The black dust particles would never go away. I could wash off the new stuff, but I still had old particles that seemed to be ground in. So I put the clay bar to work and I was probably able to get rid of 98% of the particles. Then I used the wax that came with the kit to seal it all up. I highly recommend using a clay bar - it's just time consuming. I probably spent 2.5 hours total on the car today between washing, claying, and waxing. But I should be good for a while.

A picture is below. You can see in the left side of the picture that there are not any little black spots but there are still some on the right. The left side was after using the bar and the right side was before. NOTE: Pictures are AFTER just washing the car.



This picture is a before shot - you can see some of the larger particles a little better (there are lots of smaller particles not as visible in the picure that the bar got rid of too!)



This picture is an after shot - MUCH better... =)

Comments on Next Notes Client

Thanks to Ed Brill for the great screenshots for "Hannover", the next release of the Lotus Notes Client after 7.0. Personally, I think it's going to be a great release and a huge step forward for Lotus. From other threads at Ed's site, many people have commented on an improved UI. This should silence us for a while.

After sending out a link to the screenshots, here's what some people internally have said:

- Cool - and it is looking more and more like Outlook! Guess they finally caught on!!! :-)

- Very nice! I know what Pat is going to say - kinda looks like the Outlook client we should've been using all along :) Long live Notes! Never had a problem with it and never even had to use Notes Medic. Evidence of a good product!

- Ha windows XP, er I mean Lotus Notes rules! looks liek an XP theme

- Looks like MSN Explorer!!!!

Another Sametime 6.5.1 for iSeries Hotfix

I posted the other day about a PTF Hotfix to hopefully correct an issue with my IM server crashing. According to the Sametime 6.5.1 FP1 Release Notes (I have NOT put on FP1!), this SPR SMAK62VK9B is for:

Community Services multiplexer crashes under heavy workload

This fix addresses an issue that caused the multiplexer (STMux.exe) application to crash when handling simultaneous connections from 20K to 30K users.

SPR# ISCO69SBQ4, ISCO69SBM3, SMAK62VK9B, SMAK63SE55 (ALL PLATFORMS)

While I hope this fixes the issue, I am nowhere near 20k users - currently I have only 266 logins. I'm still sticking with my theory in the previous post that it has to do with some TCP/IP stack PTFs that I applied the day before this all started.

I've sent IBM some joblogs, job stack dumps, and an NSD. Hopefully this STMUX PTF will correct it though.

Domino for iSeries Shared Memory Cleanup

We've been waiting for a way to easily clean up shared memory on the iSeries without taking down all active Domino partitions. Well, I've got two commands that IBM sent me (sshhhh, don't tell anyone...).

DLTDOMSMEM - Delete Domino Shared Memory - You specify the server name for it to clean up and run (while that partition is down, of course).

DLTDOMSEM - Delete Domino Semaphores - Same as above.

This is pretty handy if you get shared memory issues and need to clean up just that one server. Thanks to IBM for providing this - hopefully it will start shipping with Domino for iSeries releases in the future.

As with other hotfixes, etc... I post about here, you should probably contact tech support for this. However, you can also e-mail me or reply to this blog and I can get this to you.

Soft Delete Bug in Notes 6.5.3 and 6.5.4

We've been troubleshooting an issue with Soft Deletions in the mail file where the trash is emptied after just a few minutes or hours instead of our standard 48 hours.

If you're seeing this issue you should call tech support to see if this fix is for your situation. We just received the following from Lotus Tech Support:

A easy programmatic solution is to add the following code to the mail template's Database Script PostOpen event. Note: Simply disabling and re-enabling using the SetOption method does not result in the header entry being updated (probably because db is not closed between) - so the below method is the most efficient programmatic approach:

Dim hours As Integer
Set db=source.Database
If db.getoption(dbopt_softdelete) Then
hours%=db.undeleteexpiretime
db.undeleteexpiretime=hours%+1
db.undeleteexpiretime=hours%
End If

Sametime 6.5.1 for iSeries Hotfix

No, this is not the fixpack FP1 that was pulled because of issues. This is a hotfix for an issue I'm having with my Sametime (I can still call it that, right?) 6.5.1 Server for iSeries.

According to SPR HHAN5XYJDJ I should be seeing issues with my STLOGGER process and it should have been fixed in 3.1. Well, I'm apparently seeing the same call stack issues as some others with the STMUX task. About once a week my Sametime server will hang. I don't get a crash, but we just can't IM - not good with hundreds of users...

This all started after I loaded some TCP/IP stack PTFs (fixes) for my iSeries. These fixes were hopefully supposed to correct a hang I've been having with my mail server. Seems like one of the 7 or 8 fixes I loaded that day may be the culprit for this hang, but IBM hasn't confirmed that. Will load the hotfix tonight and see how it goes.

Alter Ego

To keep this blog on the topic of Tech stuff (generally Lotus/IBM), I will post non-tech stuff to a different blog. I'll try to have some fun there sometimes, but it will mostly be about politics and religion.

Check me out if you wish...


http://vastrightwing.blogdns.com

Lotus Notes 6.5.4 Rollout

On May 26 I blogged about Smart Upgrade. Here are some updated stats on my 6.5.4 rollout:

May 26
6.5.4 - 47%
6.5.3 - 26%
6.5.2 - 16%
6.5.1 - 11%
June 6
6.5.4 - 59%
6.5.3 - 21%
6.5.2 - 10%
6.5.1 - 9%

News Story

"Insurgents in Iraq have detonated a suicide bomb at a mosque, killing 15 and injuring dozens more. On top of that, countless Qurans were burned and the mosque left in shambles."

Sadly, I haven't seen this type of reporting by the media. Instead we read this and this about the American mishandling of the Quran at Gitmo. How many hundreds or thousands of copies of the Quran have the Muslim terrorists destroyed in the wake of missions that they believe that Quran told them to go on?

It's indeed tragic if Americans have mishandled the Quran at Gitmo. What's more tragic is that many of the detainees at Gitmo complaining of this would have no problem blowing up a mosque and destroying Qurans themselves.

Website Running Domino

Was looking for some song lyrics today and the first Google hit was to:

http://www.sing365.com/music/lyric.nsf/Dream-Theater...

Notice the .nsf in the path? I saw the UNID and thought it looked Notes-ish. So then I looked on to the left and saw the nsf.

According to Netcraft, it's running Domino 5.0.9 on Linux. It's a pretty quick site though - imagine it on 6.5 or even 7 with the Linux improvements 7 will have!

Journal of the Whills 3:12

OK, so only the most die-hard Star Wars fans may know what means. I had no clue until I was watching Deleted Magic last night.

The Prophecy:
"And in time of greatest despair, there shall come a savior, and he shall be known as : THE SON OF THE SUN. And he shall bring Balance to the Force."
Journal of the Whills, 3:12

Delete Magic "is a non-profit film, which will be made available for free via the internet" about Star Wars that includes "deleted scenes, alternate footage, and a look behind the scenes at what didn't make it to the final cut of the Star Wars trilogy." It contains the 3rd draft of the original opening scroll for A New Hope. The preface to that text is the prophecy regarding the one to bring Balance to the Force. Remember from Episode I that Anakin didn't have a father and would have been "the son of the sun." Well, here is the opening scroll - interesting how different it is from the one from Episode IV as we know it today:

From http://cache.cow.net/~draith/sw/scripts/SW-Starkiller3

The REPUBLIC GALACTICA is dead. Ruthless
trader barons, driven by greed and the
lust for power, have replaced enlighten-
ment with oppression, and "rule by the
people" with the FIRST GALACTIC EMPIRE.

For over a thousand years, generations of
JEDI KNIGHTS were the guardians of peace
and justice in the galaxy. Now these
legendary warriors are all but extinct.
One by one they have been hunted down and
destroyed by the sinister agents of the
Emperor: the DARK LORDS OF THE SITH.

It is a period of civil wars. Rebel
Armies, striking from fortresses hidden
deep within the Great rift, have won a
crushing victory over the powerful Imper-
ial Starfleet. The Emperor knows that
one more such defeat will bring a thou-
sand more solar systems into the rebellion,
and Imperial control of the Outland systems
could be lost forever. To crush the
rebellion once and for all, the Emperor
has sent one of his most ferocious Dark
Lords to find the secret rebel strongholds
and destroy them...

"Insufficient Memory" error Domino 6.5.3 and 6.5.4

Error "Insufficient memory - index pool is full" After NIF Global Pool Grows Very Large

Problem
The error "Insufficient memory - index pool is full" is generated on a Domino 6.5.3 or 6.5.4 server after the NIF Global Pool grows to about 128 MB. Users are not able to connect to the Domino server and the server must be rebooted.


Solution
The issue was reported to Quality Engineering as SPR #JPAS6ALJQE and a fix is being researched. A test fix has been made available for the following releases and platforms:

* Domino/Windows and Domino/AIX Release 6.5.3
* Domino/Windows and Domino/AIX Release 6.5.3 FP1
* Domino/Windows, Domino/AIX and Domin/Solaris Release 6.5.4.


Qualified customers should contact IBM Technical Support to obtain the test fix.

One particular customer collected memory dumps for two days. The first day NIF Global Pool stayed at approximately 1 MB. The second day the memory dump showed the following (the NIF global pool is approximately 2 MB in size):

+17 2 2097164 SHRMEM BLK_NIF_POOL - NIF global pool

Six hours later the memory dump showed the following (the NIF global pool is approximately 86 MB)

+17 86 90178052 SHRMEM BLK_NIF_POOL - NIF global pool

and then the memory in NIF global pool grows to approximately 128 MB:

+17 128 134218496 SHRMEM BLK_NIF_POOL - NIF global pool