Good news for the CMS@Home application


Advanced search

Message boards : News : Good news for the CMS@Home application

Author Message
ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar
Send message
Joined: 29 Aug 05
Posts: 66
Credit: 272,877
RAC: 5,897
Message 28698 - Posted: 27 Jan 2017, 20:59:36 UTC

This afternoon we demonstrated the final link in the chain of producing Monte Carlo data for CMS using this project (and the -dev project too, of course), namely the transfer of result files from the temporary Data Bridge storage to a CMS Tier 2 site's storage element (SE). To summarise, the steps are:

o Creating a configuration script defining the process(es) to be simulated
o Submitting a batch of jobs of duration and result-file size suitable for running by volunteers
o Having those jobs picked up by volunteers running BOINC and the CMS@Home application, and the result files returned to the Data Bridge
o Running "merge" jobs on a small cluster at CERN to collect the smaller files into larger files (~2.2 GB) -- this step has to be done at CERN as most volunteers will not have the bandwidth (or data plan!) to handle the data volumes required. This step also serves to a large extent as the verification step required to satisfy CMS of the result files' integrity.
o Transferring the merged files into the Grid environment where they are then readily available to CMS researchers around the world

Thanks, everybody. From here on it gets more political, but we've been garnering support as the project progressed. We now need to move into a more "production" environment and convince central powers-that-be to take over the responsibility of submitting suitable workflows and collecting the results. You will still see some changes in the future, especially as we bring some of the more-advanced features across here from the -dev project.
____________

computezrmle
Send message
Joined: 15 Jun 08
Posts: 43
Credit: 895,320
RAC: 1,418
Message 28730 - Posted: 30 Jan 2017, 13:46:36 UTC

Are there any consequences for normal volunteers, e.g.:
- new ports to be opened in the firewall
- new data bridge
- how large are the downloads/uploads
- etc. ...

ivan
Volunteer moderator
Project tester
Volunteer developer
Volunteer tester
Project scientist
Avatar
Send message
Joined: 29 Aug 05
Posts: 66
Credit: 272,877
RAC: 5,897
Message 28732 - Posted: 30 Jan 2017, 16:31:24 UTC - in response to Message 28730.

Are there any consequences for normal volunteers, e.g.:
- new ports to be opened in the firewall
- new data bridge
- how large are the downloads/uploads
- etc. ...

At the moment I don't expect firewall or data bridge changes. We would have to "educate" the production team on the range of suitable output file sizes and job CPU duration.
____________


Post to thread

Message boards : News : Good news for the CMS@Home application