Major Communication Problem

[Robin]

I had this problem last year during BETA testing and it ain't pretty.

I have songs that three coro projects sing. The two projects are on two different controllers. Last night show ran perfectly. Tonight two hours into show I have no bodies but mouths are moving. Other sections of the display are not working properly either. Last year Dan sent me a fix as it was a communication problem.

If songs are played in sequence editor there is no problem.

A little more background. I have 37 controllers and am using 576 channels. The display is spread out over about 500 feet long and 80 feet deep. Total lights are about 300,000 plus or minus.

Oh and one TV station is coming either Wed. or Thurs. and another one is coming next week. Oh forgot the paper will be here next week two but since they do not do video they don't count.

[Dale W]

You probably have one bad controller that is mucking up the others. They will test fine, but in the context of the sequences, you see peculiar behavior. It happened to me last year. I ended up taking two controllers off line and all my weirdness disappeared.


You also could have a bad cable that has an intermittent short in it - meaning, some times it works, some times it doesn't.

Welcome to my world. I have lived here (not with 37 controllers) for three years.

And this year is even weirder.

[Tim Fischer]

Since you had the same problem last year that was fixed by a software upgrade, AND it works for awhile before it messes up (assuming I've understood that correctly), I'd want to talk with Dan before I started going crazy looking for a bad comm cable that may not exist...

Does restarting the show fix the problem?

-Tim

[-klb-]

No CFL lamps anywhere in the system? No power supplies plugged in that might contain capacitors? I don't think it sounds likely in this case, but just a sanity check..

I have experienced some really strange intermittent issues with CFL ballasts impacting other controllers plugged into the same circuit...

Next, have you tried splitting out into more than one communication network? Maybe sometimes the sequence is busy enough that rather than giving lots of lag due to network traffic, it is dropping commands... I already know of another way that the sequence editor handles playing sequences differently than the show player..

Just some thoughts..

- Kevin

[Robin]

Let me address each question.

All cables are tested before they are hooked up. I have a tester as I learned that lesson two years ago.

There are no CFL's in the system.

It does not work if restarted.

During the first run of the show there is a long lag between songs. Dan told me last year that this would happen "after the fix" was installed.

All of you have the fix in the latest version of S2 as someone else had the problem during testing last year.

I am using the USB485B High Voltage booster.

The first night of the show there were no problems. Last night there were no problems for at least the first hour. The second hour when I went out is when I noticed the problem.

I do not think it is a controller issue as sequences will operate correctly if run manually with the sequence editor. (However, I did change out one of the controllers yesterday as I had a problem with a channel sticking on with the first one that was used. The back up controller was repaired this summer by Dan.) Actually that controller was replaced prior to light up so even that is not an issue.

There is no issue as to the load on either of these controllers.

There are other issues through out the system. For Example: the second controller that is in line had channels off during one of the songs in question. On another song there was no problem with that controller.

I really think this is a communication issue and not a controller issue and have already sent an email to Dan as of 8pm last night. So will update you all here. In the meantime keep brainstorming as I will be doing the show manually tonight. Let me tell you that is not fun, but people deserve to see a quality product and you do what you have to do.

Thanks

[Shubb]

I am interested to hear the outcome with this. I have having similar issues with controllers not responding correctly. I thought it was the ELL, but I am having channels not shut-off/not come on on all controllers, wire or wireless.

But I am having the same problems either through the sequence editor or show.

let me know what you find.

Scott

[rwertz]

If the sequence works in the editor, then there’s at least a reasonable chance that the problem is not communication. (But, I’m not suggesting you rule out anything.)

There have been several issues this year that sounded like communication issues that turned out to be caused by channel configuration problems. Each time the person swore their channel configuration was correct. So I would not rule that out. A partially configured channel seems to have the most unpredictable effect.

Also, if you use a background sequence you can cause trouble if you reference the same unit/circuit in both the background and main sequence.

[John]

Is it possible that when the computer is idle for a long period (no mouse or keyboard input) it is trying to go into the "Sleep" mode interrupting the running sequence?

Check the Power Options by:
Right Clicking on the Desktop
Select Properties
Screen Saver Tab at the top
Click the Power Box
Check if the Turn Off Hard Disks / System Standby / System Hibernates are all set to Never. (see attached picture)
Just a thought as a sanity check...




Attached files

[Robin]

I understand the configuration idea and the idle theory as well. I am pretty sure this is not the issue and Dan agrees at this time. We are going to try something in a few minutes so will know more after that. Will report back. Also, one thing the tip off was the fact that it works in sequence editor. That means there is a sync problem with all the channels running during the show. Actually there is a lag to some extent.

[Tim Fischer]

Good luck... I know how frustrating this kind of stuff can be (not to mention how helpless it feels when solving it is out of your hands...)

You noted that restarting the show doesn't help, but it sounds like it's working (for awhile) the following night? I wonder what makes it 'reset' so it's working again? If you could figure that out, maybe you wouldn't have to run it manually, but could just 'reset' it a few times through the night, before it started messing up... (Just trying to brainstorm a workaround...)

-Tim

[Robin]

So far so good on the fix. Will let everyone know after it runs tonight for the entire show. But here is what Dan told me to do.

Since sequences would run in the sequence editor there was no need for a repeater. What he told me to do was to set all the channels in each sequence that was "constant on" to 99% intensity. Evidently this forced the program to make a lot of commands and for some reason it seems to be working. Like I said will reserve verdict until end of tonight but will be sure to keep you all updated. Thanks for the help and maybe this will help someone else in the future.

Robin

[Robin]

Well have a problem with one sequence so will check it after the show is over. But the other sequences seem to be working properly.

[rwertz]

Glad to hear you’ve got things straightened out.

Did you change your intensity to 99% for the entire sequence? I remember a post where Dan suggested changing between 100 and 99% every couple seconds to force more commands to overcome a possible missed command.

[Robin]

I changed a cell every second or so for 20 seconds.

[Robin]

FIXED!

so far