| jmlowe_ | https://jetstream-cloud.org/archive/publications.php | 00:44 |
|---|---|---|
| jmlowe_ | masber ^^^^ | 00:46 |
| masber | jmlowe_, you have vms flavors with 44 cores? | 01:02 |
| masber | do you use NUMA? | 01:02 |
| jmlowe_ | yes | 01:02 |
| jmlowe_ | 44 hyperthreaded cores | 01:02 |
| jmlowe_ | 24 real cores | 01:02 |
| jmlowe_ | 2 socket dell m630 blades | 01:04 |
| masber | why not using ironic for those use cases where the tenant needs a whole machine? | 01:07 |
| jmlowe_ | we don't use openvswitch or sdn | 01:08 |
| jmlowe_ | we also wanted the flexibility to live migrate and take maintenance that way | 01:08 |
| jmlowe_ | linpack gets %97 of bare metal for our purposes that was close enough | 01:09 |
| jmlowe_ | we aren't exactly doing hpc, we are targeting users that don't currently use nationally funded resources | 01:10 |
| jmlowe_ | %97 of users eligible to use NSF funded resources don't | 01:11 |
| masber | ok | 01:11 |
| jmlowe_ | so the tradeoffs we made were performance for uptime | 01:11 |
| masber | you don't use sdn means neutron talks to your physical switches and routers to configure the networking? | 01:12 |
| jmlowe_ | all vxlan | 01:12 |
| jmlowe_ | linuxbridge, so encapsulated tenant networks and neutron doesn't talk to any switches | 01:13 |
| *** b1airo has joined #scientific-wg | 04:22 | |
| *** b1airo has quit IRC | 04:56 | |
| *** simon-AS559 has joined #scientific-wg | 06:44 | |
| *** simon-AS559 has quit IRC | 06:49 | |
| *** simon-AS559 has joined #scientific-wg | 06:49 | |
| *** simon-AS559 has quit IRC | 07:11 | |
| *** simon-AS559 has joined #scientific-wg | 07:11 | |
| *** priteau has joined #scientific-wg | 08:24 | |
| *** simon-AS559 has quit IRC | 09:27 | |
| *** simon-AS559 has joined #scientific-wg | 09:29 | |
| *** priteau has quit IRC | 09:51 | |
| *** simon-AS559 has quit IRC | 11:03 | |
| *** priteau has joined #scientific-wg | 11:26 | |
| *** b1airo has joined #scientific-wg | 11:28 | |
| *** rbudden has joined #scientific-wg | 13:00 | |
| *** simon-AS559 has joined #scientific-wg | 13:02 | |
| *** b1airo has quit IRC | 13:28 | |
| *** simon-AS559 has quit IRC | 13:32 | |
| *** trandles has joined #scientific-wg | 13:50 | |
| bollig | jmlowe_: looking forward to your writeup on luminous. blynch has some data on our rebuilds that might be of interest. we have dual NVMe and 6 SSDs per server, and our rebuilds have been pretty quick. I’ll remind him to bring the data and notes to the SC BoF. | 14:16 |
| *** simon-AS559 has joined #scientific-wg | 14:17 | |
| bollig | masber: jmlowe_: we run panasas on our HPCs, but opted to run a separate Ceph cluster for OpenStack. a) it is significantly cheaper and scales well; b) we partition it for dual purpose (block and object); c) it reserves the parallel I/O performance for nodes on the HPC that can actually use it (panasas requires a kernel mod we don’t have on hypervisors); d) ceph as a separate island allows us to satisfy use-cases like controlled-access data which | 14:23 |
| bollig | don’t fit well on global namespace filesystems. | 14:23 |
| bollig | masber: if you’re interested in scheduler integrations, you might look at Adapative Computing’s MOAB. They have/had a cloud add-on that provisions compute nodes to alleviate queue pressure; scale-in is included. We opted out of that feature due to cost. Also, at one point I hacked together a few scripts to report openstack resource availability to the PBS scheduler, and matching prologue and epilogue scripts to manage VM provisioning for indivi | 14:28 |
| bollig | pipeline jobs. I can share more details of that, but it would be operating in an unsupported mode. | 14:28 |
| trandles | jmlowe_: it's official now https://www.openstack.org/summit/berlin-2018 | 17:25 |
| jmlowe_ | bollig masber: I saw an early demo of the moab openstack integration, it was pretty slick if you want to move the partition back and forth between traditional batch and cloud as needed | 18:52 |
| jmlowe_ | Whew, just finished recovering the last pg, took roughly 6 days | 18:53 |
| *** priteau has quit IRC | 19:15 | |
| *** priteau has joined #scientific-wg | 19:37 | |
| *** priteau has quit IRC | 19:41 | |
| *** priteau has joined #scientific-wg | 19:59 | |
| *** simon-AS559 has quit IRC | 20:45 | |
| *** oneswig has joined #scientific-wg | 20:56 | |
| *** martial has joined #scientific-wg | 21:00 | |
| *** b1airo has joined #scientific-wg | 21:06 | |
| *** StefanPaetowJisc has joined #scientific-wg | 21:23 | |
| priteau | There was no time for AOB in the meeting, I would like to share something else: I am happy to announce that Chameleon is being renewed for another three years! https://ci.uchicago.edu/blog/cloud-computing-testbed-chameleon-renewed-second-phase | 22:04 |
| jmlowe_ | excellent, congratultions | 22:04 |
| martial | congratulations Pierre :) | 22:04 |
| oneswig | Bravo priteau, looking forward to Chameleon's Second Age! | 22:05 |
| martial | I should also introduce a new federation effort backed by NIST and IEEE | 22:05 |
| priteau | Looking forward to continue contributing to OpenStack :-) | 22:05 |
| martial | titled P2302 ... I know catchy :) | 22:05 |
| b1airo | great news priteau | 22:05 |
| jmlowe_ | I just heard from Jeff Adams a few minutes ago, we now have 4 speakers for SC'17 booth talks | 22:06 |
| oneswig | martial: got a link? | 22:06 |
| martial | oneswig: lookibg | 22:06 |
| martial | You can find more details at http://collaborate.nist.gov/twiki-cloud-computing/bin/view/CloudComputing/FederatedCloudPWGFC and http://sites.ieee.org/sagroups-2302/ | 22:06 |
| martial | we can discuss this more next week | 22:07 |
| oneswig | Sounds good to me | 22:07 |
| martial | for now not much happening there ... the kick off meeting was last week | 22:07 |
| *** oneswig has quit IRC | 22:08 | |
| StefanPaetowJisc | Sorry folks, as much as there was a suggestion I should try to get to Sydney to bring up macOS support for Moonshot, I've been so snowed under here it's not possible. Next OS summit, honest | 22:08 |
| StefanPaetowJisc | :-/ | 22:08 |
| martial | wait and see:) | 22:10 |
| jmlowe_ | Please excuse my ignorance, what's Moonshot? | 22:10 |
| jmlowe_ | damn, we had a meeting today didn't we? | 22:11 |
| jmlowe_ | nothing like a vomiting 3yr old to reoder your priorities | 22:11 |
| *** b1airo has quit IRC | 22:13 | |
| *** priteau has quit IRC | 22:15 | |
| StefanPaetowJisc | Moonshot is a GSSAPI mechanism for federated non-web authentication :-) | 22:16 |
| StefanPaetowJisc | If you're used to the Grid Computing world, you'll know GSI-SSH. It does something similar, but just with grid certs | 22:16 |
| StefanPaetowJisc | https://wiki.moonshot.ja.net - macOS was a priority for us in the end when we bet on the wrong horse (Windows instead of macOS). | 22:17 |
| StefanPaetowJisc | :-/ | 22:17 |
| *** StefanPaetowJi-1 has joined #scientific-wg | 22:19 | |
| *** StefanPaetowJisc has quit IRC | 22:19 | |
| *** StefanPaetowJi-1 is now known as StefanPaetowJisc | 22:19 | |
| trandles | GRRRR, sorry I missed the meeting today | 22:21 |
| *** StefanPaetowJisc has quit IRC | 22:22 | |
| trandles | jmlowe_: quick link ;) http://eavesdrop.openstack.org/meetings/scientific_wg/2017/scientific_wg.2017-09-05-21.00.log.html | 22:25 |
| jmlowe_ | ah, interesting, I'll have to check it out | 22:26 |
| jmlowe_ | My contribution for the meeting would have been using senlin to backfill with osg instances, using webhooks to scale up and down as needed | 22:32 |
| martial | mike: save it for next time :) | 22:35 |
| *** martial has quit IRC | 22:35 | |
| *** b1airo has joined #scientific-wg | 23:30 | |
| *** b1airo has quit IRC | 23:47 | |
| *** b1airo has joined #scientific-wg | 23:50 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!