SciLINC

Background info here available

Applications for Win and Lin. (Though no WUs are being send atm)

From Background info:

Missouri Botanical Garden proposes Scientific Literature Indexing on Networked Computers (SciLINC), a public-resource computing application that will automatically index large amounts of digitized scientific literature, ultimately providing users with an integrated Web portal, available at www.botanicus.org, for the discovery of information about plants.

Grüße vom Sänger

Cheers Saenger

Free Tibet/ Tibet Libre Team Forum

... analyze text from digitized botanical literature ...

Sever Page shows 57,882 wu's to send, but BOINC gets "No work from Project"

http://www.scilinc.org/SciLINC/server_status.php

Sever Page shows 57,882 wu's to send, but BOINC gets "No work from Project"

http://www.scilinc.org/SciLINC/server_status.php

Grüße vom Sänger

Sever Page shows 57,882 wu's to send, but BOINC gets "No work from Project"

SciLINC will be sending new wu by Monday, June 18. Apologies for the delay!
Chris Freeland

SciLINC will be sending new wu by Monday, June 18. Apologies for the delay!
Chris Freeland

Grüße vom Sänger

And THX for the WU (btw: you say "new wu", that's singular, does this mean there will be only one available? )

I was going to report I tried to contact another person about SciLINC and got a response, but I see he, Chris Freeland, Has already posted here.

What he didn't mention was, the other guy, Ron Parker, went on vacation, which is sort of what I guessed. Figures things go foobar when no one is watching the store.

Thanks Chris for the email.

I just got a bunch of work. But then the server went down, and many of those tasks are stuck in downloading and uploading status. Each one took about a second to run.

It occurs to me that,

1) all the work takes only about a second to run

and

2) the leadtime is only 1 day, which means all results downloaded will go into EDF almost immediately, and will return results immediately.

So...their site will be HAMMERED with all the upload/download requests. I wonder if their site is up for that kind of traffic? Perhaps that is why it went down almost immediately after offering units.

I had 153 before for some reason they stopped being sent. They all ran 2 to 5 seconds. Windows OK, Linux had all errors. My credit totals 0.68 for that.

Can anyone else attest to this project possibly causing BOINC daemon memory/CPU usage to balloon? Saying this because of the combination of just receiving many SciLINC WU's and now over a thousand files located in the project directory. I posted to BOINC forums.

Personally I had three computers hooked up to SciLINC and I only noticed something funny when I was checking the weather. The PC acted sluggish and the download was slow. This is on my Dual core machine with a 5MB cable modem, I should not feel anything slow.

After fianlly pausing BOINC I noticed that Boinc was still taking 97% cpu power at all tiomes. It took me several Boinc Shutdowns to reset the scilinc project only after I locked my firewall. Once I reset, I detached the project. Finally my pc is working right.

I do not know about you but I will wait on this project to get a forum and find out what the &$%% is going on. I still can't clear my laptop, I had to pull the battery.

Holy crap! It completely locked up 3 dual core machines. No amount of the usual remedies would solve the problem. Something got completely so FUBAR, I had to uninstall BOINC, trash the remaining BOINC folder, reboot, and reinstall BOINC. What a cluster (!*&!

I had several (of the same) projects running on each of those machines.

So after the surgery, I attached only one of each project on each machine. Yep. It's this one.

So...I still have copies of the BOINC folders from just before I NUKED them.

Here's my problem. I has one Seasonal Attribution Project job on each machine, only about *3 days* away from completion. AAARGH! Any way to recover them? Little help?

SciLINC locked up my computer also. It downloaded 50 wu's, each taking 6 seconds to calculate on my P3 1GHz.

But in total its been busy/locked up for 8 hours!

Yeah, FUBAR describes it quite well!

I suspended all other projects, set SciLINC to no new work. Now 8 hours later, everythings back to normal again!

My main host downloaded 50 wu's too. But I wont get there within the next 8 hours... *fingers crossed* It has 2 SAPs also (35% and 5%)...

My NEW BOINC-Site

Why people joined BOINC Synergy...

So far I'm attributing this to the huge number of files each WU uses/generates. My project directory reached a peak of nearly 10,000 files, and BOINC had to keep track of every single one of them. The issue is that while BOINC is still doing work, it'll be next to impossible to communicate with it via RPC. However, looking at stdoutdae.txt showed that it was still transferring files. With at least a hundred files per WU, its obviously going to take some time for BOINC to get them transferred out, despite each one being only a kilobyte or so in size. So I'm going to let it cool down for the next several hours with every other project suspended and SciLINC set to no new work. I had to stop BOINC and manually edit client_state.xml to do this. Hopefully SciLINC server will be up long enough.

Lesson learned is:

Do NOT attach to SciLINC!

Otherwise it will take over BOINC and you will be forced to either uninstall it or it will steal time from you and your other projects.

To admins: The solution may be to ZIP up files before transferring them. I believe its just the vast number of file transfers BOINC is trying to handle thats grinding it to a halt. If everything is in a zip BOINC will have far less to deal with and reduces server overhead. But please test before handing out the WUs to everyone, this is a very major and easy to catch problem!

Ok, I see I'm not the only one who had main problems with the up-/dowloading queue. Also the BOINCmanager used over 40MB RAM when it tried to handle this zillions of files.

I finally managed to "kill" BOINC via Windows TaskManager after 30 minutes of upload on dsl for the first 5 or 6 WUs. After restarting BOINC I suspended network and detached. The upload of over 20 WUs of this project would have taken hours I guess. And I won't even imagine how that hurts dial-upper's.

@SciLINC admins: please reduce the number of files for down-/uploading significantly and/or use ZIP as AlphaLaser suggested below!
AND please open your message board for reporting such bugs!

Greetings from Cori

FWIW, I managed to keep the impact of returning SciLINC results to a minimum by increasing the number of total file transfers from 8 to 16 as well as the number of file transfers per project from 2 to the same 16 (using BoincView). This way, the BOINC client flushes the queue quickly and does not page fault like mad as before and it doesn't hog the system anymore.

HTH

FWIW, I managed to keep the impact of returning SciLINC results to a minimum by increasing the number of total file transfers from 8 to 16 as well as the number of file transfers per project from 2 to the same 16 (using BoincView). This way, the BOINC client flushes the queue quickly and does not page fault like mad as before and it doesn't hog the system anymore.

HTH

I left in run to see how bad it could get.

I found that just running the boinc.exe was better. If I had BOINC manager running it was un-responsive and BOINCview would not update this host. With just the core client BOINCview will update sometimes.

I had done the same thing, but set to max of 8 transfers and only have SciLINC running, all others suspended. I tried 16, but 8 seems better for me. I finally got through all the work. Something like 163 tasks and now that the downloads have stopped and all work done its just uploads left. I was up to something like 5,260 max transfers at one point, and its been running for at least 4 hours since I modified it. IT ran all night in slow motion mode. This is one host.

I want to see how long it takes, from now to clear the last 5,000 uploads.

So this message time represents the 5,000 mark.

How do you do that ?

I noted on the thread I posted to the BOINC forums that it took about 16-17 hours for the host to fully recover from the situation. I'm guessing the host pulled between 100-150 WUs, I'd check the results page on SciLINC but it appears down at the moment. BOINCView had trouble contacting the host so I just changed settings by editing files. I pushed my settings to 24/24. I might try an experiment of re-enabling SciLINC with these new transfer settings and monitor closely to see if it is good enough to reduce the backlog that occurred in the first place.

How do you do that ?

Using the "Open BOINC core-client configuration" button in the "Hosts" toolbar of BoincView (http://boincview.amanheis.de/).

With 16/16 the system is quite responsive and as soon as it gets ahold of the SciLINC server, it depletes the results queue in seconds.

HTH

So this message time represents the 5,000 mark.

Holy crap! It completely locked up 3 dual core machines. No amount of the usual remedies would solve the problem. Something got completely so FUBAR, I had to uninstall BOINC, trash the remaining BOINC folder, reboot, and reinstall BOINC. What a cluster (!*&!

I had several (of the same) projects running on each of those machines.

So after the surgery, I attached only one of each project on each machine. Yep. It's this one.

So...I still have copies of the BOINC folders from just before I NUKED them.

Here's my problem. I has one Seasonal Attribution Project job on each machine, only about *3 days* away from completion. AAARGH! Any way to recover them? Little help?

Regards,

Wimtea

/me dawns his flame proof fire suit.

Hello all,

I am the one responsible for SciLINC and the problems you have been experiencing.

First of all I would like to apologize for the problems everyone is seeing. When I returned Friday I realized that workunits were not flowing, the forums were disabled and an unexpected number of people and teams had joined the project. I would also like to say thank you to everyone here that has responded with helpful input for diagnosing and correcting these issues.

My first priorities were getting the work flowing again then turning on the forums so that there would be a place for feedback, venting and frustration resolution (that almost sounds politically correct.)

Just to let you all know, we did perform internal testing on the project and did not see these issues. Of the machines we tested, one had problems and that was my Windows development machine. The OpenGL driver was causing problems and the machine lagged a little bit. Nothing like what has been reported here. But that machine is also heavily loaded in a number of ways.

Of course the latency of the Internet is much higher than our local network. Unfortunately, I did not foresee that contributing to these problems.

Updates will be posted to the main SciLINC page as we get our database back on-line and are able to address the work unit issue.

Part of the discussion here has hints on improving SciLINC performance so that your machine may recover better once the database is back up.

Sincerely and regretfully,

Ron Parker

thx for taking the time to post an update here. far to ment admins do not take thise extra effert, and it is appriciated.

now, when will willy add new projects?

As soon as the site (stats) is stable again after all the DB changes.

Please do not PM, IM or email me for support (they will go unread/ignored). Use the forum for support.

ahh, sorry, I hadn�t noticed any probs...

Any estimates? (days, weeks, "when it�s done"

So this message time represents the 5,000 mark.

My satellite connection is fast download (up to 1.5M or better in burst mode) but slow 128k on uploads.

Host #1
At 8/8 It took about 3 hours to clear 1200 uploads.

I then set it back to 16/16, It took about 2.5 hours to clear the remaining 3800 uploads.

Around 1200 left I was able to restart other projects running.

ahh, sorry, I hadn�t noticed any probs...

Any estimates? (days, weeks, "when it�s done"

Please do not PM, IM or email me for support (they will go unread/ignored). Use the forum for support.

months! :shock:

ok, well, good to know. I wont be wating for BAM! support to start some of these new projects then.

Thx for the quick update and the good work you do for us!

Thanks for the status report admin. I' look forward to crunching the project once all the details are worked out

They had to acknowledge that the current application was nort really DC-able. They have stopped for now (but have plans for the future.

There is a very good post about it in the forum over there.

Grüße vom Sänger

I see that SciLINC has been added to BOINCstats. Yes!

Can it please be added to BAM! too?

Hi: When you are ready for more testers please send me an invite code to Mark Reiss at merko_1@juno.com -- Thank You

The project devs are asking for our input about whether to recommence the project or abandon BOINC in favour of another platform (not even sure whether the alternate platform is DC).

Free Tibet/ Tibet Libre Team Forum