Index :: BAM! General :: Task Deadlines Question

Pages: [1]
Author
Message
indigodream
PM
BAM!-ID: 22989
Joined: 2007-04-11
Posts: 2
Credits: 86,840
World-rank: 739,979
Posted 2012-05-10 09:58:32
I've recently set up an EeePC as a media server using Twonky and thought I may as well have Boinc running as the machine will be running 24/7. I was a bit concerned about the extra power consumption of Boinc using 100% of the CPU, but it turns out it's only another 3 watts!

However, I'm a bit concerned that the work being done on some tasks may be a waste of time as the deadlines may be missed. I have a current task with around 270 hours left to run with a deadline of 21 May - I think that'll be a bit tight!

So my question is, is there anyway to control which tasks are downloaded by taking the task's deadline into consideration? I.e. set BAM up so it does not download a large task with a deadline which my EeePC will probably miss?

On a side note, I have 5 tasks running at the moment. If one tasks says it will take 100 hours to complete, does it mean it will take 100 hours to complete if that task gets 100% of the processor at all times, or is task switching taken into account when calculating the remaining time? If it isn't, then the EeePC will definitely miss the 21 May deadline!

Many thanks in advance,
Barry
Odd-Rod
PM
Tester
BAM!-ID: 45166
Joined: 2008-01-31
Posts: 1578
Credits: 48,907,825
World-rank: 9,457
Posted 2012-05-10 14:01:51
Let me answer your last question first.
That 100 hours is 100 hours of actual processing time, so two tasks, each with 100 hours to go, will take a total of 200 hours before they are both completed. But Boinc's task switching may mean that one of them will finish long before the other. (There are too many factors that could influence the switching for me to even pretend I know it all!)

To address your concern, that 270 hours may not be an accurate estimate. Since this is possibly the first Work Unit (WU) you're crunching from that project it could even be a very inaccurate estimate! Boinc uses a Duration Correction Factor (DCF) to correct for wrong estimates. If a WU takes longer than was estimated (by the project, by the way), then the factor is adjusted upwards to fully compensate for the difference. When you first add a project the DCF is 100. As an example. if a WU starts out expecting to take 3 hours and actually takes 6 then the DCF for that project will be changed to 200. Note, however, that reducing the DCF is done much slower to avoid an unexpectedly short WU causing a too low DCF. In other words, if it expects 6 hours and takes only 3 hours, the DCF would not go from say, 100 to 50, but perhaps to only 90 (I don't know the exact amount it drops by, but I seem to remember reading about 10%). This avoids having more WUs downloaded than can be completed in the required time.
Since this all affects the estimated times, I hope you see why I answered the last question first!

To answer the first question. If the DCF of the project in question is too low, it will be adjusted, but only after you let the WU finish. What you may like to do is after a day or so, see if the remaining time has come down enough that you might finish in time.

Wow, I thought I was going to 'quickly' reply to your posting!

When Boinc is left alone for a long period it eventually seems to sort it self out, but you may miss a deadline or two while it does so
Not so bad for a 1 hour WU, but for 270 hours...
indigodream
PM
BAM!-ID: 22989
Joined: 2007-04-11
Posts: 2
Credits: 86,840
World-rank: 739,979
Posted 2012-05-10 14:05:56
Odd-Rod wrote:
Let me answer your last question first.
That 100 hours is 100 hours of actual processing time, so two tasks, each with 100 hours to go, will take a total of 200 hours before they are both completed. But Boinc's task switching may mean that one of them will finish long before the other. (There are too many factors that could influence the switching for me to even pretend I know it all!)

To address your concern, that 270 hours may not be an accurate estimate. Since this is possibly the first Work Unit (WU) you're crunching from that project it could even be a very inaccurate estimate! Boinc uses a Duration Correction Factor (DCF) to correct for wrong estimates. If a WU takes longer than was estimated (by the project, by the way), then the factor is adjusted upwards to fully compensate for the difference. When you first add a project the DCF is 100. As an example. if a WU starts out expecting to take 3 hours and actually takes 6 then the DCF for that project will be changed to 200. Note, however, that reducing the DCF is done much slower to avoid an unexpectedly short WU causing a too low DCF. In other words, if it expects 6 hours and takes only 3 hours, the DCF would not go from say, 100 to 50, but perhaps to only 90 (I don't know the exact amount it drops by, but I seem to remember reading about 10%). This avoids having more WUs downloaded than can be completed in the required time.
Since this all affects the estimated times, I hope you see why I answered the last question first!

To answer the first question. If the DCF of the project in question is too low, it will be adjusted, but only after you let the WU finish. What you may like to do is after a day or so, see if the remaining time has come down enough that you might finish in time.

Wow, I thought I was going to 'quickly' reply to your posting!

When Boinc is left alone for a long period it eventually seems to sort it self out, but you may miss a deadline or two while it does so
Not so bad for a 1 hour WU, but for 270 hours...


Thanks for your reply. I think I'll let Boinc do its thing and see if it sorts itself out.

Barry
Odd-Rod
PM
Tester
BAM!-ID: 45166
Joined: 2008-01-31
Posts: 1578
Credits: 48,907,825
World-rank: 9,457
Posted 2012-05-10 19:19:36
barrydrink wrote:

Thanks for your reply. I think I'll let Boinc do its thing and see if it sorts itself out.

Barry


In general, Boinc does seem to get it right eventually. And remember, 'eventually' means after crunching quite a few WUs for each project that the host is connected to - which may be many weeks if the projects have long WUs. But one case it will never be able to get right, is if a particular (slow) host cannot finish a WU in time even if it crunches full time. That's when that project needs to be detached from the host.

Just an additional thought from some of my Boincing experience. When adding a new host don't add too many projects at once. If the DCF is too low on a few projects you will get too much work and miss some deadlines. Of course, this tip may be a bit late now...

Hopefully in your case Boinc will sort itself out.
CRSouser
PM
BAM!-ID: 22419
Joined: 2007-04-02
Posts: 9
Credits: 14,943,935
World-rank: 24,494
Posted 2014-03-27 12:41:33
Odd-Rod wrote:

Hopefully in your case Boinc will sort itself out.

I personally haven't seen it all sort itself out, quite the opposite.

I recently purchased a new computer and added the client first thing.. it processes 8 units at a time and has the fastest processing time of all of my hosts working on projects. Then all of a sudden started happening is I was receiving no credit from that host for two weeks on ANY projects.

So I started to investigate and found what was happening is World Community Grid especially, I've also started seeing seen it on Rosetta, started giving it large sets of downloads that sometimes had a deadline of less than a week or even less than a day (and in some cases less than 12 hours).

Being they all had less than 7 day processing deadline, they paused work on all other projects and worked these projects at High Priority. Then especially on ones with less than 24 hours response time it would return the completed result and the results were getting rejected and no credit applied due to past deadline.

All of my computers process most standard work sets in 3-5 hours and they process 4-8 tasks each, so I still should be getting credit. I've tried also doing a work around of setting my communication with BAM to a short interval and I tend to get "Communication Deferred". Other projects also fall down in the queue for so long as new force fed work units from projects like WCU prevent them from running that it creates a cycle of things running at high priority and deadlines getting missed as not everything can run High Priority. What I've had to start doing is manually monitoring each host and marking projects like WCU temporarily as "Get no new Tasks" to try to attempt this from happening. Yet despite doing this I have a Climate Prediction work unit for example on one host that has received no computing time in 2+ weeks (it is one of the 500hr work units too from CP) due to this practice and Climate Prediction in my BAM manager priority list is one of my highest. WCU is half of what my next project is and it still gets "High Priority". I was using the default weighting in the BOINCStats console using ranges from basically 50-200 but read in another thread that they are seeing WCU always send "300" no matter what you have it set to which might explain something..so as another work around I went in and adjusted my priorities of my 5 projects across all of them so they collectively add up to 10000 (the max collective value of the weighting) to see if that makes a difference.

I approached support on World Community Grid's site and on Social Media about their short processing deadlines and how it was affecting my other projects and got blown off. I suspect WCU is still actually getting the results but we are not getting in credit for them.

I feel this practice of short deadlines by WCU and growing number of other projects realizing what is happening and competing for resources may also be doing is not in the spirit of the open grid community and not being able to set preferences in BAM is a failure of the BAM manager. It then causes me as a user have to micro-manage my projects on each host to ensure I am going to complete them all on time or that my projects are actually being worked in the priority I have set.

At this point I STRONGLY want a preference in the BAM Manager to be able to say not to give me any tasks with a deadline less than X days in general; even better on a project by project.
NATE1
PM
BAM!-ID: 101099
Joined: 2011-06-02
Posts: 277
Credits: 33,406,708,657
World-rank: 120
Posted 2014-03-27 16:38:50
CRSouser wrote:

I feel this practice of short deadlines by WCU and growing number of other projects realizing what is happening and competing for resources


you got that right...
even Dr. Dave is concerned over the dwindling number of volunteers and hosts
and yes as the number go down the project admins are starting to pull all kinds of things to try to get as much use of your hosts as they can.
even at the expense of other projects.
short deadlines, shorter that before
tasks being download with remaining run time in the hundreds if not thousands of hours
or remaining run time so short that the host downloads too much work and after the 1st task get done it go into running high pri.. status just on that projects. even force feeding work to the hosts, host ask for 1 second of work, which should be 1 task and the project server sends 2 or more.
so the Competition for resources is starting to really heat up.
which in turn is causing volunteers to pull the pull at an even faster rate.
therefor causing even less resources and so on and so on and so on.......
noderaser
PM Profile Donator
BAM!-ID: 13859
Joined: 2006-12-03
Posts: 780
Credits: 136,942,552
World-rank: 4,236
Posted 2014-03-28 02:13:56
I would recommend that you not set the communication with BAM preference to a short interval, as BAM has nothing to do with sending you work--it only updates your project list and some preferences when you make changes to them from the BAM site, and setting a short interval there unnecessarily increases BAM traffic.

One thing you should note about the new host, is that it may take a few cycles of work before BOINC manager figures out how quickly your system is completing work, and therefore how much work it asks for from the projects. That's where the "Connect to network about every" and "Maintain enough work for an additional" days settings under your Work preferences come into play. Those settings affect how much work gets downloaded. Due to the length of the ClimatePrediction tasks, you will likely see most other work get priority until you get closer to the deadline and it gets a higher priority. I'm not sure why you are getting a lot of other work if CPDN has a high resource share, but the BOINC manager will ultimately try and adjust the total computation time for each project to something that resembles your resource share. Again, the long length of CPDN work units may be throwing the calculations for this off a bit, but it will get better over time. Every time you attach a new project, you'll also typically get work from it (even if you don't need additional work) unless you immediately select "No new work".

Regarding not getting credit from WCG for completed work, you should check your Results at their site and see if you have a lot of tasks "Pending Validation". Because they're using their own system, they don't get validated as quickly as other BOINC projects and you can get a substantial lag between returning completed work and receiving credit.

Regarding the use of short deadlines, you'd have to ask each project about their criteria for deadlines. It largely depends on how the results are being used, and the kinds of returns they receive.

Pages: [1]

Index :: BAM! General :: Task Deadlines Question
Reason: