| *** brinzhang_ has joined #openstack-meeting | 00:06 | |
| *** tobiash has quit IRC | 00:08 | |
| *** brinzhang has quit IRC | 00:09 | |
| *** tobiash has joined #openstack-meeting | 00:11 | |
| *** diablo_rojo has joined #openstack-meeting | 00:15 | |
| *** yamamoto has joined #openstack-meeting | 00:18 | |
| *** yamamoto has quit IRC | 00:23 | |
| *** aspiers has quit IRC | 00:35 | |
| *** jamesmcarthur has joined #openstack-meeting | 00:46 | |
| *** Liang__ has joined #openstack-meeting | 00:57 | |
| *** Liang__ is now known as LiangFang | 00:59 | |
| *** LiangFang has quit IRC | 01:01 | |
| *** Liang__ has joined #openstack-meeting | 01:01 | |
| *** jamesmcarthur has quit IRC | 01:01 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:09 | |
| *** yamamoto has joined #openstack-meeting | 01:16 | |
| *** jawad_axd has joined #openstack-meeting | 01:21 | |
| *** nanzha has joined #openstack-meeting | 01:22 | |
| *** ekcs has quit IRC | 01:25 | |
| *** jawad_axd has quit IRC | 01:26 | |
| *** ykatabam has joined #openstack-meeting | 01:35 | |
| *** lseki has quit IRC | 01:42 | |
| *** igordc has quit IRC | 01:42 | |
| *** jawad_axd has joined #openstack-meeting | 01:42 | |
| *** Xuchu has joined #openstack-meeting | 01:44 | |
| *** jawad_axd has quit IRC | 01:47 | |
| *** jawad_axd has joined #openstack-meeting | 02:03 | |
| *** larainema has joined #openstack-meeting | 02:04 | |
| *** jawad_axd has quit IRC | 02:07 | |
| *** ekcs has joined #openstack-meeting | 02:08 | |
| *** ekcs has quit IRC | 02:15 | |
| *** brinzhang has joined #openstack-meeting | 02:21 | |
| *** brinzhang_ has quit IRC | 02:24 | |
| *** ekcs has joined #openstack-meeting | 02:26 | |
| *** markvoelker has joined #openstack-meeting | 02:36 | |
| *** markvoelker has quit IRC | 02:41 | |
| *** ekcs has quit IRC | 02:43 | |
| *** igordc has joined #openstack-meeting | 02:45 | |
| *** ekcs has joined #openstack-meeting | 02:55 | |
| *** brinzhang_ has joined #openstack-meeting | 03:03 | |
| *** yamamoto has quit IRC | 03:07 | |
| *** yamamoto has joined #openstack-meeting | 03:07 | |
| *** yamamoto has quit IRC | 03:07 | |
| *** brinzhang has quit IRC | 03:07 | |
| *** yamamoto has joined #openstack-meeting | 03:07 | |
| *** ykatabam has quit IRC | 03:09 | |
| *** artom has quit IRC | 03:16 | |
| *** brinzhang has joined #openstack-meeting | 03:17 | |
| *** brinzhang_ has quit IRC | 03:21 | |
| *** ekcs has quit IRC | 03:21 | |
| *** ekcs has joined #openstack-meeting | 03:29 | |
| *** Xuchu has quit IRC | 03:31 | |
| *** Xuchu has joined #openstack-meeting | 03:32 | |
| *** ykatabam has joined #openstack-meeting | 03:35 | |
| *** Guest14611 has quit IRC | 03:50 | |
| *** benj has joined #openstack-meeting | 03:51 | |
| *** benj is now known as Guest85345 | 03:51 | |
| *** ekcs has quit IRC | 03:54 | |
| *** ekcs has joined #openstack-meeting | 03:57 | |
| *** ekcs has quit IRC | 03:57 | |
| *** ricolin has joined #openstack-meeting | 03:57 | |
| samP | #startmeeting masakari | 04:00 |
|---|---|---|
| openstack | Meeting started Tue Oct 29 04:00:20 2019 UTC and is due to finish in 60 minutes. The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot. | 04:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 04:00 |
| *** openstack changes topic to " (Meeting topic: masakari)" | 04:00 | |
| openstack | The meeting name has been set to 'masakari' | 04:00 |
| samP | Hi all for masakari | 04:00 |
| *** tashiromt has joined #openstack-meeting | 04:02 | |
| *** kiyofujin has joined #openstack-meeting | 04:03 | |
| samP | Any one has issues, agenda or topics to discuss today's meeting? | 04:08 |
| samP | seems like no team members are in the meeting today. | 04:08 |
| tashiromt | Hi | 04:09 |
| samP | tashiromt: Hi | 04:09 |
| *** nitinuikey has joined #openstack-meeting | 04:09 | |
| *** tpatil has joined #openstack-meeting | 04:10 | |
| tpatil | samP: Sorry to join late today | 04:10 |
| samP | tpatil: NP, thanks for joining | 04:12 |
| samP | tpatil: any issues need to discuss? | 04:12 |
| *** yamamoto has quit IRC | 04:12 | |
| *** yamamoto has joined #openstack-meeting | 04:12 | |
| *** yamamoto has quit IRC | 04:13 | |
| *** yamamoto has joined #openstack-meeting | 04:13 | |
| *** nitinuikey56 has joined #openstack-meeting | 04:13 | |
| *** nitinuikey56 has quit IRC | 04:13 | |
| *** nitinuikey has quit IRC | 04:14 | |
| *** nitinuikey has joined #openstack-meeting | 04:14 | |
| tpatil | No show-stopper issues | 04:17 |
| *** brinzhang_ has joined #openstack-meeting | 04:17 | |
| samP | tpatil: OK, got it. | 04:17 |
| *** yamamoto has quit IRC | 04:17 | |
| nitinuikey | samP: I have sent project status ppt to present in the summit. Please share your feedback. | 04:18 |
| samP | I got the slides for summit session. I will try to send my comments today. | 04:19 |
| samP | nitinuikey: Thanks. ^^ | 04:19 |
| nitinuikey | samP: Thanks | 04:19 |
| samP | if no other items to discuss, let's finish today's meeting here. | 04:19 |
| samP | Thank you all for attending! | 04:19 |
| tpatil | Ok | 04:20 |
| samP | Thanks all! | 04:21 |
| samP | #endmeeting | 04:21 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 04:21 | |
| openstack | Meeting ended Tue Oct 29 04:21:11 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 04:21 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-10-29-04.00.html | 04:21 |
| samP | 12 | 04:21 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-10-29-04.00.txt | 04:21 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-10-29-04.00.log.html | 04:21 |
| *** brinzhang has quit IRC | 04:21 | |
| *** ykatabam has quit IRC | 04:21 | |
| *** yamamoto has joined #openstack-meeting | 04:26 | |
| *** tpatil has quit IRC | 04:26 | |
| *** ykatabam has joined #openstack-meeting | 04:28 | |
| *** tashiromt has left #openstack-meeting | 04:32 | |
| *** nitinuikey has quit IRC | 04:34 | |
| *** browny has quit IRC | 04:47 | |
| *** kiyofujin has quit IRC | 04:57 | |
| *** markvoelker has joined #openstack-meeting | 04:58 | |
| *** vishalmanchanda has joined #openstack-meeting | 05:02 | |
| *** markvoelker has quit IRC | 05:03 | |
| *** brinzhang has joined #openstack-meeting | 05:18 | |
| *** brinzhang_ has quit IRC | 05:21 | |
| *** igordc has quit IRC | 05:31 | |
| *** dviroel has quit IRC | 05:37 | |
| *** igordc has joined #openstack-meeting | 05:38 | |
| *** sridharg has joined #openstack-meeting | 05:38 | |
| *** links has joined #openstack-meeting | 05:38 | |
| *** brinzhang_ has joined #openstack-meeting | 05:46 | |
| *** markvoelker has joined #openstack-meeting | 05:46 | |
| *** brinzhang has quit IRC | 05:49 | |
| *** ekcs has joined #openstack-meeting | 05:50 | |
| *** markvoelker has quit IRC | 05:56 | |
| *** markvoelker has joined #openstack-meeting | 05:56 | |
| *** dtrainor_ has quit IRC | 05:56 | |
| *** brinzhang has joined #openstack-meeting | 05:59 | |
| *** markvoelker has quit IRC | 06:00 | |
| *** dtrainor has joined #openstack-meeting | 06:01 | |
| *** brinzhang_ has quit IRC | 06:02 | |
| *** diablo_rojo has quit IRC | 06:11 | |
| *** jawad_axd has joined #openstack-meeting | 06:14 | |
| *** brinzhang has quit IRC | 06:17 | |
| *** brinzhang has joined #openstack-meeting | 06:17 | |
| *** ekcs has quit IRC | 06:25 | |
| *** diablo_rojo has joined #openstack-meeting | 06:26 | |
| *** Luzi has joined #openstack-meeting | 06:38 | |
| *** brinzhang_ has joined #openstack-meeting | 06:48 | |
| *** brinzhang_ has quit IRC | 06:49 | |
| *** brinzhang_ has joined #openstack-meeting | 06:49 | |
| *** brinzhang has quit IRC | 06:51 | |
| *** igordc has quit IRC | 06:52 | |
| *** keiko-k has joined #openstack-meeting | 06:58 | |
| *** diablo_rojo has quit IRC | 07:03 | |
| *** jawad_axd has quit IRC | 07:04 | |
| *** nanzha has quit IRC | 07:04 | |
| *** nanzha has joined #openstack-meeting | 07:08 | |
| *** e0ne has joined #openstack-meeting | 07:10 | |
| *** e0ne_ has joined #openstack-meeting | 07:11 | |
| *** e0ne has quit IRC | 07:11 | |
| *** e0ne_ has quit IRC | 07:12 | |
| *** e0ne has joined #openstack-meeting | 07:14 | |
| *** jawad_axd has joined #openstack-meeting | 07:15 | |
| *** jawad_ax_ has joined #openstack-meeting | 07:16 | |
| *** e0ne has quit IRC | 07:17 | |
| *** jawad_axd has quit IRC | 07:20 | |
| *** brinzhang has joined #openstack-meeting | 07:27 | |
| *** joxyuki has joined #openstack-meeting | 07:27 | |
| *** ykatabam has quit IRC | 07:27 | |
| *** rcernin has quit IRC | 07:28 | |
| *** brinzhang_ has quit IRC | 07:30 | |
| *** pcaruana has joined #openstack-meeting | 07:37 | |
| *** slaweq has joined #openstack-meeting | 07:42 | |
| *** dkushwaha has joined #openstack-meeting | 07:43 | |
| *** tesseract has joined #openstack-meeting | 07:49 | |
| *** trident has quit IRC | 07:55 | |
| *** rsimai has joined #openstack-meeting | 07:55 | |
| *** trident has joined #openstack-meeting | 08:01 | |
| dkushwaha | #startmeeting tacker | 08:01 |
| openstack | Meeting started Tue Oct 29 08:01:59 2019 UTC and is due to finish in 60 minutes. The chair is dkushwaha. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:02 |
| *** openstack changes topic to " (Meeting topic: tacker)" | 08:02 | |
| openstack | The meeting name has been set to 'tacker' | 08:02 |
| joxyuki | hi | 08:03 |
| dkushwaha | #topic Roll Call | 08:03 |
| *** openstack changes topic to "Roll Call (Meeting topic: tacker)" | 08:03 | |
| dkushwaha | hello joxyuki | 08:03 |
| joxyuki | hello dkushwaha | 08:03 |
| *** takahashi-tsc has joined #openstack-meeting | 08:04 | |
| dkushwaha | look no other members available today | 08:04 |
| takahashi-tsc | Hi | 08:04 |
| joxyuki | ok | 08:04 |
| dkushwaha | hello takahashi-tsc | 08:05 |
| joxyuki | hi, takahashi-san | 08:05 |
| *** kopecmartin|off is now known as kopecmartin | 08:05 | |
| keiko-k | hi | 08:06 |
| dkushwaha | hi keiko-k | 08:06 |
| dkushwaha | Anybody have issues, agenda or topics to discuss today's meeting? | 08:06 |
| joxyuki | about PTG | 08:07 |
| joxyuki | Did you make a etherpad for PTG? | 08:07 |
| joxyuki | or I can do it. | 08:07 |
| dkushwaha | joxyuki, yes, I had created link and sent mail on ML regarding same last week | 08:08 |
| dkushwaha | #link https://etherpad.openstack.org/p/Tacker-PTG-Ussuri | 08:08 |
| joxyuki | dkushwaha, thanks | 08:08 |
| dkushwaha | I had added some topics from my side. | 08:08 |
| dkushwaha | Please feel free to update/add topics in the same | 08:09 |
| joxyuki | yes, I will add my topics later. | 08:09 |
| dkushwaha | joxyuki, ok | 08:10 |
| takahashi-tsc | I also add my topic later. | 08:10 |
| *** yaawang has quit IRC | 08:10 | |
| dkushwaha | takahashi-tsc, thanks | 08:10 |
| *** yaawang has joined #openstack-meeting | 08:10 | |
| joxyuki | And when will we able to start Tacker PTG? | 08:10 |
| joxyuki | dkushwaha, you have Project update session at 6th 12:05pm-12:20pm | 08:12 |
| joxyuki | and I have my presentation at 6th 10:50am-11:30am | 08:13 |
| joxyuki | I think we can start the PTG from 13pm or later. | 08:13 |
| dkushwaha | joxyuki, For PTG we has sufficient time, we have full day on Wednesday(9:00am to 4:30pm), and halfday on Thursday(9:00am to 12:30pm) | 08:13 |
| *** yamamoto has quit IRC | 08:13 | |
| *** brinzhang_ has joined #openstack-meeting | 08:13 | |
| *** yamamoto has joined #openstack-meeting | 08:15 | |
| dkushwaha | joxyuki, Aah, I missed timing of your session. But anyway, ok we will start PTG after your session(after13:00) | 08:15 |
| joxyuki | ok, got it | 08:16 |
| dkushwaha | I will also notify the timing on ML | 08:16 |
| *** nitinuikey has joined #openstack-meeting | 08:16 | |
| *** tpatil has joined #openstack-meeting | 08:16 | |
| takahashi-tsc | It's good for me too. (after 13:00) | 08:17 |
| *** brinzhang has quit IRC | 08:17 | |
| dkushwaha | takahashi-tsc, nice, I hope no other timing conflict there. | 08:17 |
| dkushwaha | i have no update from my side | 08:19 |
| joxyuki | no more from me | 08:19 |
| tpatil | dkushwaha: Hi | 08:19 |
| dkushwaha | hello tpatil | 08:19 |
| tpatil | I want to discuss about one patch | 08:19 |
| dkushwaha | sure | 08:20 |
| tpatil | #link : https://review.opendev.org/#/c/618086 | 08:20 |
| tpatil | since now we have force_delete option available with admin user, is it possible to disallow delete vnf if it's in PENDING_CREATE state? | 08:20 |
| dkushwaha | tpatil, force_delete will admin privilege. So VNF in such abnormal state can be deleted with admin only(with force_delete), and not allowed to normal users | 08:22 |
| dkushwaha | tpatil, returning 409 is expected behavior as in https://review.opendev.org/#/c/618086 | 08:24 |
| dkushwaha | untill we not use force_delete with admin user | 08:25 |
| tpatil | ok | 08:25 |
| dkushwaha | i will put my comment on patch too | 08:26 |
| tpatil | ok thank you | 08:26 |
| dkushwaha | Do we have anything else? otherwise we can close this meeting | 08:28 |
| dkushwaha | tpatil, JFYI: etherpad link for PTG: https://etherpad.openstack.org/p/Tacker-PTG-Ussuri | 08:30 |
| tpatil | dkushwaha: Ok, I will check, thanks | 08:31 |
| *** hyunsikyang has joined #openstack-meeting | 08:33 | |
| dkushwaha | Thanks Folks. Lets close this meeting | 08:33 |
| dkushwaha | excited to meet you all in PTG | 08:33 |
| dkushwaha | #endmeeting | 08:34 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 08:34 | |
| openstack | Meeting ended Tue Oct 29 08:34:01 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 08:34 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-10-29-08.01.html | 08:34 |
| *** brinzhang has joined #openstack-meeting | 08:34 | |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-10-29-08.01.txt | 08:34 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-10-29-08.01.log.html | 08:34 |
| *** brinzhang_ has quit IRC | 08:37 | |
| *** ralonsoh has joined #openstack-meeting | 08:44 | |
| *** ykatabam has joined #openstack-meeting | 08:46 | |
| *** nitinuikey has quit IRC | 08:46 | |
| *** priteau has joined #openstack-meeting | 08:46 | |
| *** dkushwaha has quit IRC | 08:47 | |
| *** brinzhang_ has joined #openstack-meeting | 08:49 | |
| *** tpatil has quit IRC | 08:49 | |
| *** brinzhang has quit IRC | 08:51 | |
| *** Luzi has quit IRC | 09:07 | |
| *** rpittau|afk is now known as rpittau | 09:08 | |
| *** Luzi has joined #openstack-meeting | 09:22 | |
| *** yamamoto has quit IRC | 09:23 | |
| *** Liang__ has quit IRC | 09:23 | |
| *** yamamoto has joined #openstack-meeting | 09:30 | |
| *** Xuchu has quit IRC | 09:32 | |
| *** yamamoto has quit IRC | 09:35 | |
| *** e0ne has joined #openstack-meeting | 09:36 | |
| *** keiko-k has quit IRC | 09:41 | |
| *** ociuhandu has joined #openstack-meeting | 09:47 | |
| *** ociuhandu has quit IRC | 09:50 | |
| *** brinzhang has joined #openstack-meeting | 09:59 | |
| *** ykatabam has quit IRC | 10:01 | |
| *** brinzhang_ has quit IRC | 10:02 | |
| *** ykatabam has joined #openstack-meeting | 10:03 | |
| *** pcaruana has quit IRC | 10:03 | |
| *** lpetrut has joined #openstack-meeting | 10:07 | |
| *** priteau has quit IRC | 10:20 | |
| *** pcaruana has joined #openstack-meeting | 10:24 | |
| *** jamesmcarthur has quit IRC | 10:27 | |
| *** artom has joined #openstack-meeting | 10:36 | |
| *** bbowen has quit IRC | 10:38 | |
| *** rsimai has quit IRC | 10:47 | |
| *** rsimai has joined #openstack-meeting | 10:56 | |
| *** rfolco|ruck has quit IRC | 11:04 | |
| *** rfolco|ruck has joined #openstack-meeting | 11:04 | |
| *** ykatabam has quit IRC | 11:11 | |
| *** ianychoi__ is now known as ianychoi | 11:24 | |
| *** raildo has joined #openstack-meeting | 11:36 | |
| *** yamamoto has joined #openstack-meeting | 11:38 | |
| *** Lucas_Gray has joined #openstack-meeting | 11:50 | |
| *** ociuhandu has joined #openstack-meeting | 11:51 | |
| *** yamamoto has quit IRC | 11:52 | |
| *** ociuhandu has quit IRC | 11:55 | |
| *** markvoelker has joined #openstack-meeting | 11:59 | |
| *** brinzhang_ has joined #openstack-meeting | 12:02 | |
| *** markvoelker has quit IRC | 12:03 | |
| *** brinzhang has quit IRC | 12:05 | |
| *** markvoelker has joined #openstack-meeting | 12:05 | |
| *** bbowen has joined #openstack-meeting | 12:09 | |
| *** lseki has joined #openstack-meeting | 12:16 | |
| *** rfolco|ruck is now known as rfolco|rover | 12:17 | |
| *** larainema has quit IRC | 12:22 | |
| *** belmoreira has joined #openstack-meeting | 12:27 | |
| *** yamamoto has joined #openstack-meeting | 12:32 | |
| *** pcaruana has quit IRC | 12:45 | |
| *** Wryhder has joined #openstack-meeting | 12:51 | |
| *** munimeha1 has quit IRC | 12:51 | |
| *** Lucas_Gray has quit IRC | 12:52 | |
| *** Wryhder is now known as Lucas_Gray | 12:52 | |
| *** yamamoto has quit IRC | 12:55 | |
| *** dklyle has quit IRC | 12:57 | |
| *** david-lyle has joined #openstack-meeting | 12:57 | |
| *** pcaruana has joined #openstack-meeting | 12:58 | |
| *** enriquetaso has joined #openstack-meeting | 13:01 | |
| *** njohnston_ has joined #openstack-meeting | 13:03 | |
| *** njohnston_ has quit IRC | 13:03 | |
| *** liuyulong has joined #openstack-meeting | 13:11 | |
| *** sridharg has quit IRC | 13:11 | |
| *** Lucas_Gray has quit IRC | 13:13 | |
| *** david-lyle is now known as dklyle | 13:28 | |
| *** mriedem has joined #openstack-meeting | 13:29 | |
| *** vishalmanchanda has quit IRC | 13:32 | |
| *** yamamoto has joined #openstack-meeting | 13:37 | |
| *** e0ne has quit IRC | 13:42 | |
| *** e0ne has joined #openstack-meeting | 13:42 | |
| *** joxyuki has quit IRC | 13:48 | |
| *** pcaruana has quit IRC | 13:52 | |
| *** pcaruana has joined #openstack-meeting | 13:55 | |
| *** Luzi has quit IRC | 13:55 | |
| *** lajoskatona has joined #openstack-meeting | 13:59 | |
| *** shintaro has joined #openstack-meeting | 14:01 | |
| haleyb | hi | 14:01 |
| lajoskatona | Hi | 14:01 |
| bcafarel | o/ | 14:01 |
| * haleyb will start ovn convergenge in one minute, need to find meeting link with all the #instructions | 14:02 | |
| lajoskatona | haleyb: https://wiki.openstack.org/wiki/Network/Meetings I think this is the link | 14:02 |
| haleyb | #startmeeting networking-ovn | 14:02 |
| openstack | Meeting started Tue Oct 29 14:02:52 2019 UTC and is due to finish in 60 minutes. The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:02 |
| *** openstack changes topic to " (Meeting topic: networking-ovn)" | 14:02 | |
| *** diablo_rojo has joined #openstack-meeting | 14:02 | |
| openstack | The meeting name has been set to 'networking_ovn' | 14:02 |
| bcafarel | hi again then :) | 14:03 |
| *** brinzhang_ has quit IRC | 14:03 | |
| *** brinzhang_ has joined #openstack-meeting | 14:04 | |
| haleyb | hi, i'll wait for a few more, although i know there's a downstream meeting at the same time | 14:04 |
| *** brinzhang_ has quit IRC | 14:05 | |
| *** dklyle has quit IRC | 14:05 | |
| bcafarel | + potential dst effect | 14:06 |
| *** brinzhang_ has joined #openstack-meeting | 14:06 | |
| haleyb | #topic Announcements | 14:06 |
| *** openstack changes topic to "Announcements (Meeting topic: networking-ovn)" | 14:06 | |
| *** brinzhang_ has quit IRC | 14:07 | |
| *** dklyle has joined #openstack-meeting | 14:07 | |
| haleyb | I am all out of sorts so far today, trying to find a link to the ptg etherpads, but basically we will be discussint this topic there as well | 14:08 |
| *** e0ne has quit IRC | 14:09 | |
| haleyb | https://etherpad.openstack.org/p/Shanghai-Neutron-Planning | 14:09 |
| bcafarel | https://etherpad.openstack.org/p/Shanghai-Neutron-Planning ? | 14:09 |
| bcafarel | :) | 14:09 |
| haleyb | https://etherpad.openstack.org/p/neutron-shanghai-forum-brainstorming | 14:09 |
| haleyb | so for anyone that attends we can discuss in-person as well | 14:10 |
| haleyb | #topic ML2-OVS-DVR/OVN convergence | 14:10 |
| *** openstack changes topic to "ML2-OVS-DVR/OVN convergence (Meeting topic: networking-ovn)" | 14:10 | |
| haleyb | https://etherpad.openstack.org/p/ML2-OVS-OVN-Convergence | 14:10 |
| haleyb | we've added to the list over the past two weeks | 14:11 |
| *** e0ne has joined #openstack-meeting | 14:11 | |
| *** dklyle has quit IRC | 14:12 | |
| haleyb | the first thing on the list is to update the devstack plugin code in the neutron tree so it can also install OVN | 14:13 |
| haleyb | I will try and get that started this week, since next week is busy | 14:13 |
| *** eharney has quit IRC | 14:14 | |
| haleyb | before i continue, should i re-order the etherpad and make it obvious what the high-level work items are? that way others can help out | 14:15 |
| * haleyb wonders if he put others to sleep :) | 14:16 | |
| lajoskatona | haleyb: we are listening (reading actually) | 14:17 |
| haleyb | lajoskatona: ack | 14:18 |
| haleyb | i will make a more obvious task list on the etherpad. it originally started as a brainstorming session, so i'll clean it up | 14:19 |
| *** takahashi-tsc has quit IRC | 14:19 | |
| lajoskatona | haleyb: next week we will discuss it anyway and I suppose there will be changes :-) | 14:20 |
| *** e0ne has quit IRC | 14:20 | |
| haleyb | lajoskatona: yes, there could be some. and i would encourage others to add things they think are missing, or help clarify what is there | 14:21 |
| *** e0ne has joined #openstack-meeting | 14:23 | |
| bcafarel | sorry got sidetacked by a deployment, +1 on sorting (and showing clear possible pending tasks) | 14:23 |
| haleyb | one thing liuyulong had a question on last week was the IPv6 status in OVN, and i'm happy to say there's been an IPv6 Prefix Delegation change proposed to OVN core repository | 14:23 |
| haleyb | https://mail.openvswitch.org/pipermail/ovs-dev/2019-October/363896.html | 14:26 |
| haleyb | since we don't have any reviews up to talk about, i'll just open it up for discussion | 14:28 |
| haleyb | #topic Open Discussion | 14:28 |
| *** openstack changes topic to "Open Discussion (Meeting topic: networking-ovn)" | 14:28 | |
| liuyulong | haleyb, good to know, so before that is there any substitute? | 14:28 |
| haleyb | liuyulong: not for prefix delegation, no | 14:29 |
| *** Liang__ has joined #openstack-meeting | 14:31 | |
| haleyb | if there are no other topics i can end the meeting | 14:32 |
| haleyb | ok, thanks for attending, and see some of you in person next week, safe travels! | 14:33 |
| haleyb | #endmeetin | 14:33 |
| haleyb | #endmeeting | 14:33 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 14:33 | |
| openstack | Meeting ended Tue Oct 29 14:33:57 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:34 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/networking_ovn/2019/networking_ovn.2019-10-29-14.02.html | 14:34 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/networking_ovn/2019/networking_ovn.2019-10-29-14.02.txt | 14:34 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/networking_ovn/2019/networking_ovn.2019-10-29-14.02.log.html | 14:34 |
| lajoskatona | o/ | 14:34 |
| *** lajoskatona has left #openstack-meeting | 14:34 | |
| *** dviroel has joined #openstack-meeting | 14:34 | |
| *** jamesmcarthur has joined #openstack-meeting | 14:38 | |
| *** ykatabam has joined #openstack-meeting | 14:44 | |
| *** shintaro has quit IRC | 14:50 | |
| *** yamamoto has quit IRC | 14:51 | |
| *** jamesmcarthur has quit IRC | 14:53 | |
| *** slaweq has quit IRC | 14:54 | |
| *** slaweq has joined #openstack-meeting | 14:55 | |
| *** Liang__ has quit IRC | 14:57 | |
| *** gyee has joined #openstack-meeting | 14:57 | |
| *** enriquetaso has quit IRC | 14:59 | |
| *** eharney has joined #openstack-meeting | 14:59 | |
| *** Liang__ has joined #openstack-meeting | 14:59 | |
| *** ociuhandu has joined #openstack-meeting | 15:00 | |
| *** jamesmcarthur has joined #openstack-meeting | 15:01 | |
| *** dklyle has joined #openstack-meeting | 15:01 | |
| *** links has quit IRC | 15:02 | |
| *** ociuhandu has quit IRC | 15:07 | |
| *** ociuhandu has joined #openstack-meeting | 15:12 | |
| *** liuyulong has quit IRC | 15:17 | |
| *** Liang__ has quit IRC | 15:19 | |
| *** ociuhandu has quit IRC | 15:20 | |
| *** ociuhandu has joined #openstack-meeting | 15:22 | |
| *** jawad_ax_ has quit IRC | 15:24 | |
| *** jawad_axd has joined #openstack-meeting | 15:25 | |
| *** mattw4 has joined #openstack-meeting | 15:25 | |
| *** ociuhandu has quit IRC | 15:26 | |
| *** yamamoto has joined #openstack-meeting | 15:27 | |
| *** jawad_axd has quit IRC | 15:29 | |
| *** yamamoto has quit IRC | 15:44 | |
| *** slaweq has quit IRC | 15:49 | |
| *** slaweq has joined #openstack-meeting | 15:54 | |
| *** nanzha has quit IRC | 15:58 | |
| *** nanzha has joined #openstack-meeting | 15:59 | |
| slaweq | #startmeeting neutron_ci | 16:00 |
| slaweq | hi | 16:00 |
| openstack | Meeting started Tue Oct 29 16:00:37 2019 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| ralonsoh | hi | 16:00 |
| njohnston | o/ | 16:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 16:00 | |
| openstack | The meeting name has been set to 'neutron_ci' | 16:00 |
| bcafarel | o/ | 16:01 |
| *** igordc has joined #openstack-meeting | 16:01 | |
| slaweq | ok, we already have usual participants so we can start | 16:01 |
| slaweq | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 16:02 |
| slaweq | please open now | 16:02 |
| slaweq | sorry, I had to take care of daughter | 16:04 |
| slaweq | but now I'm back | 16:04 |
| slaweq | #topic Actions from previous meetings | 16:04 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 16:04 | |
| slaweq | first one | 16:05 |
| slaweq | ralonsoh to check if there is any possibility to do somethig like ovsdb-monitor for openflows | 16:05 |
| ralonsoh | one sec | 16:05 |
| ralonsoh | #link https://review.opendev.org/#/c/689150/ | 16:05 |
| ralonsoh | this is a patch to implement a OF monitor | 16:05 |
| ralonsoh | then, if approved, we can use it in UT/FT testing | 16:05 |
| slaweq | and You want to use it only in the tests? | 16:06 |
| ralonsoh | for now I don't see any other use | 16:06 |
| ralonsoh | we raised this bug just for debugging tests | 16:06 |
| ralonsoh | but of course, could be use anywhere | 16:06 |
| slaweq | yes, but I wonder if we couldn't simply run ovs-ofctl monitor as external service and log it's output to some file | 16:07 |
| slaweq | similary to how now dstat is working in ci jobs | 16:07 |
| slaweq | wouldn't that be enough? | 16:07 |
| ralonsoh | well you need a bit of logix there to clean the output | 16:08 |
| ralonsoh | but sure this can be done | 16:08 |
| ralonsoh | the goal of this class is to be used anywhere you want to track the OF changes | 16:09 |
| slaweq | ok, this monitor is run "per bridge" so it would be better to run it "per test" or "per test class" probably | 16:09 |
| ralonsoh | yes, only per bridge | 16:09 |
| ralonsoh | cmd = ['ovs-ofctl', 'monitor', bridge_name, 'watch:', '--monitor'] | 16:09 |
| slaweq | so python is better for that, right :) | 16:10 |
| ralonsoh | I think so | 16:10 |
| slaweq | thx ralonsoh, I will add this patch to my review queue | 16:10 |
| ralonsoh | thanks | 16:10 |
| slaweq | thx for working on this, that may help us a lot with debugging some failed tests in the future | 16:11 |
| slaweq | ok, next action | 16:12 |
| slaweq | ralonsoh to investigate failed fullstack tests for dhcp agent rescheduling | 16:12 |
| ralonsoh | as commented in https://bugs.launchpad.net/neutron/+bug/1799555/comments/23 | 16:12 |
| openstack | Launchpad bug 1799555 in neutron "Fullstack test neutron.tests.fullstack.test_dhcp_agent.TestDhcpAgentHA.test_reschedule_network_on_new_agent timeout" [High,Fix committed] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 16:12 |
| ralonsoh | more time spent in reviewing logs than implementing the patch | 16:12 |
| ralonsoh | just a timout issue | 16:12 |
| ralonsoh | #link https://review.opendev.org/#/c/689550/ | 16:12 |
| ralonsoh | that's all | 16:13 |
| slaweq | yeah, I saw Your comment and patch which looks fine for me | 16:13 |
| ralonsoh | cool | 16:13 |
| slaweq | njohnston: bcafarel: please also review it if You will have some time :) | 16:13 |
| njohnston | will do | 16:13 |
| slaweq | njohnston: thx | 16:14 |
| slaweq | ok, next one | 16:14 |
| slaweq | slaweq to investigate failed neutron.tests.fullstack.test_qos.TestDscpMarkingQoSOvs | 16:14 |
| bcafarel | sure thing, added to the list :) | 16:14 |
| slaweq | unfortunatelly I didn't have time for this | 16:14 |
| slaweq | I will try to do it this (or next week), maybe somewhere at the airport/airplane | 16:14 |
| slaweq | #action slaweq to investigate failed neutron.tests.fullstack.test_qos.TestDscpMarkingQoSOvs | 16:14 |
| slaweq | bcafarel: also thx :) | 16:15 |
| slaweq | ok, next one | 16:15 |
| slaweq | slaweq to check strange "EVENT OVSNeutronAgentOSKenApp->ofctl_service GetDatapathRequest send_event" log lines in neutron.tests.functional.agent.l2.extensions.test_ovs_agent_qos_extension.TestOVSAgentQosExtension.test_port_creation_with_dscp_marking | 16:15 |
| slaweq | for this one I also didn't had time | 16:15 |
| slaweq | sorry for that | 16:15 |
| slaweq | #action slaweq to check strange "EVENT OVSNeutronAgentOSKenApp->ofctl_service GetDatapathRequest send_event" log lines in neutron.tests.functional.agent.l2.extensions.test_ovs_agent_qos_extension.TestOVSAgentQosExtension.test_port_creation_with_dscp_marking | 16:15 |
| slaweq | next one | 16:16 |
| slaweq | ralonsoh to try to log flows at the end of faliled functional test | 16:16 |
| ralonsoh | yes | 16:16 |
| ralonsoh | once the previous patch is approved | 16:16 |
| ralonsoh | #link https://review.opendev.org/#/c/689150/10 | 16:16 |
| ralonsoh | then we can use it for the tests | 16:17 |
| ralonsoh | one question | 16:17 |
| ralonsoh | this can be used during the test, logging the OF changes | 16:17 |
| ralonsoh | or just when the test fails, we can print the OF changes in the log | 16:17 |
| *** e0ne has quit IRC | 16:17 | |
| ralonsoh | thoughts? | 16:17 |
| slaweq | I think that logging during the test is better for debugging | 16:18 |
| ralonsoh | perfect | 16:18 |
| slaweq | as than You potentially can find out some race conditions | 16:18 |
| ralonsoh | easier, just make use of the class and when needed, log the OF changes | 16:18 |
| ralonsoh | that's all | 16:18 |
| *** igordc has quit IRC | 16:18 | |
| slaweq | ok, thx ralonsoh | 16:19 |
| slaweq | so You will take care of this, right? | 16:19 |
| ralonsoh | sure | 16:19 |
| slaweq | can You maybe open launchpad bug for it? Just for tracking purpose | 16:19 |
| ralonsoh | yes of course | 16:19 |
| slaweq | thx | 16:19 |
| slaweq | #action ralonsoh to open LP about adding OF monitor to functional tests | 16:20 |
| slaweq | ok, I think we can move on to the next topic than | 16:20 |
| slaweq | #topic Stadium projects | 16:20 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 16:20 | |
| slaweq | according to tempest-plugin-migration I don't have any updates | 16:20 |
| slaweq | do You have anything related to stadium projects? | 16:21 |
| njohnston | should we start tracking py3 support removal and zuul v3 job conversion goals in etherpads (either new or reused)? | 16:21 |
| njohnston | Or should we wait until after Shanghai for those to be finalized? | 16:21 |
| slaweq | I think we can start doing that, based on what was anounced recently by gmann | 16:22 |
| slaweq | but even we want to start it "now" it will be effectively after ptg :) | 16:22 |
| njohnston | true, true :-) | 16:22 |
| njohnston | ok so maybe let's start talking about it the meeting after PTG | 16:23 |
| bcafarel | unless everything is completed by the time we fly back :) | 16:23 |
| slaweq | njohnston: will You prepare such etherpad maybe to track progress? | 16:23 |
| slaweq | bcafarel: yeah :) | 16:23 |
| njohnston | #action njohnston prepare etherpad to track stadium progress for zuul v3 job definition and py2 support drop | 16:23 |
| slaweq | thx njohnston | 16:23 |
| slaweq | ok, lets move on than | 16:24 |
| slaweq | #topic Grafana | 16:24 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 16:24 | |
| slaweq | I have only one thing to say about grafana | 16:25 |
| slaweq | infra had issue with disk space for few days | 16:25 |
| slaweq | so we are missing data from last few days | 16:25 |
| slaweq | it started working yesterday evening | 16:25 |
| slaweq | now our numbers there looks fine but we don't have too much of recent data to check in fact | 16:26 |
| njohnston | yeah, tough to make judgements | 16:26 |
| slaweq | exactly | 16:27 |
| slaweq | I see one thing, we should remove py27 jobs from it as we removed them from queues | 16:27 |
| slaweq | I will do it today | 16:27 |
| slaweq | #action slaweq to send patch to remove py27 jobs from grafana | 16:27 |
| njohnston | +1 | 16:27 |
| slaweq | anything else You want to add/ask about grafana? | 16:28 |
| njohnston | nope | 16:29 |
| ralonsoh | no | 16:29 |
| bcafarel | let's just wait for it to fill again :) | 16:29 |
| slaweq | ok, so lets move on than | 16:29 |
| slaweq | #topic fullstack/functional | 16:29 |
| *** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)" | 16:29 | |
| slaweq | I found few new issues for You :) | 16:29 |
| slaweq | first one is something which we saw in the past already | 16:30 |
| slaweq | Problem with "AttributeError: 'str' object has no attribute 'content_type'" error agan, see | 16:30 |
| slaweq | https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_12f/690908/1/check/neutron-functional/12f8c37/controller/logs/dsvm-functional-logs/neutron.tests.functional.agent.linux.test_l3_tc_lib.TcLibTestCase.test_get_filter_id_for_ip.txt.gz | 16:30 |
| ralonsoh | this one is very curious | 16:30 |
| ralonsoh | because this is happening in the test case result return | 16:30 |
| njohnston | weird | 16:31 |
| ralonsoh | IMO, this is something related to testtools (but I can't confirm) | 16:31 |
| *** nanzha has quit IRC | 16:31 | |
| ralonsoh | btw, I don't know if the test case failed or not (this is a good indicator to know what is in content) | 16:32 |
| slaweq | yeah, it is strange | 16:32 |
| slaweq | but if You look at http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22AttributeError%3A%20'str'%20object%20has%20no%20attribute%20'content_type'%5C%22 | 16:32 |
| slaweq | it seems that this happens only on neutron functional/fullstack jobs :/ | 16:32 |
| ralonsoh | ok, so we have a pattern, pointing to us.... | 16:32 |
| slaweq | yep | 16:33 |
| slaweq | I will open LP for this one | 16:33 |
| slaweq | as it happens from time to time | 16:33 |
| slaweq | maybe we will find fix for it | 16:33 |
| *** ekcs has joined #openstack-meeting | 16:33 | |
| slaweq | or some volunteer for work on this than :) | 16:34 |
| ralonsoh | I can take a look | 16:34 |
| slaweq | #action slaweq to open LP related to "AttributeError: 'str' object has no attribute 'content_type'" error | 16:34 |
| slaweq | ralonsoh: thx, I will ping You with link to LP when I will open it | 16:34 |
| ralonsoh | thanks | 16:34 |
| slaweq | if You will have time, would be great if You can check it | 16:34 |
| slaweq | thx a lot | 16:35 |
| *** jamesmcarthur_ has joined #openstack-meeting | 16:35 | |
| slaweq | next failure which I found is | 16:36 |
| slaweq | https://0c68218832dc7cac70c7-9752c849fa19cb3b4ae0f2b2e19d3d65.ssl.cf2.rackcdn.com/691710/2/check/neutron-functional/15b9600/testr_results.html.gz | 16:36 |
| slaweq | but in this case in test's log I don't see anything obvious | 16:36 |
| slaweq | https://0c68218832dc7cac70c7-9752c849fa19cb3b4ae0f2b2e19d3d65.ssl.cf2.rackcdn.com/691710/2/check/neutron-functional/15b9600/controller/logs/dsvm-functional-logs/neutron.tests.functional.agent.test_dhcp_agent.DHCPAgentOVSTestCase.test_good_address_allocation.txt.gz | 16:36 |
| ralonsoh | slaweq, maybe | 16:37 |
| ralonsoh | in assert_good_allocation_for_port | 16:37 |
| ralonsoh | what we need to do is to retrieve the interface name and run the dhclient in the wait loop | 16:37 |
| ralonsoh | not only to check the ip list | 16:37 |
| *** tesseract has quit IRC | 16:38 | |
| slaweq | but when You run dhclient it will wait for dhcp reply and retry couple of times, no? | 16:38 |
| slaweq | so it should be like "wait loop" already | 16:38 |
| *** jamesmcarthur has quit IRC | 16:39 | |
| ralonsoh | I can take a closer look to those logs tomorrow | 16:39 |
| *** enriquetaso has joined #openstack-meeting | 16:40 | |
| *** ekcs has quit IRC | 16:40 | |
| slaweq | thx ralonsoh, but please don't waste too much of Your time on it, I saw it only once for now in the gate | 16:41 |
| ralonsoh | ok | 16:41 |
| slaweq | ok, lets move on | 16:41 |
| slaweq | fullstack tests now | 16:41 |
| slaweq | I found one new bug https://bugs.launchpad.net/neutron/+bug/1850292 | 16:41 |
| openstack | Launchpad bug 1850292 in neutron "Fullstack test can try to use IP address from outside the subnet's allocation pool" [Medium,Confirmed] - Assigned to Slawek Kaplonski (slaweq) | 16:41 |
| slaweq | I have almost working patch for it already | 16:41 |
| slaweq | but it doesn't seems for me like something very urgent as it don't hits as often | 16:43 |
| slaweq | that's all about functional/fullstack on my side | 16:43 |
| slaweq | do You have anything else to add/ask? | 16:43 |
| ralonsoh | no | 16:43 |
| *** ekcs has joined #openstack-meeting | 16:44 | |
| slaweq | ok, lets move on than | 16:44 |
| slaweq | #topic Tempest/Scenario | 16:44 |
| njohnston | not on this topic | 16:44 |
| *** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)" | 16:44 | |
| *** davidsha has joined #openstack-meeting | 16:44 | |
| slaweq | njohnston: not on which topic? | 16:44 |
| njohnston | slaweq: was responding to "do You have anything else to add/ask?" just slowly | 16:44 |
| slaweq | ahh, ok | 16:45 |
| slaweq | :) | 16:45 |
| slaweq | ok, going back to scenario tests | 16:45 |
| slaweq | I found that Multicast scenario test is failing often | 16:45 |
| slaweq | so I reported bug https://bugs.launchpad.net/neutron/+bug/1850288 | 16:45 |
| openstack | Launchpad bug 1850288 in neutron "scenario test test_multicast_between_vms_on_same_network fails" [Critical,Confirmed] - Assigned to Slawek Kaplonski (slaweq) | 16:45 |
| slaweq | and I proposed to make it unstable for now https://review.opendev.org/691855 | 16:45 |
| slaweq | but later I will want to investigate why it is failing from time to time | 16:46 |
| slaweq | I also saw some various connectivity issues, like: | 16:47 |
| slaweq | https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3ae/682483/1/check/tempest-slow-py3/3ae72c6/testr_results.html.gz | 16:47 |
| slaweq | and | 16:47 |
| slaweq | https://2a0fd24cd939d3b06641-34d5a3d9e2a9ca67eb62c8365f7602e7.ssl.cf5.rackcdn.com/679813/3/check/tempest-slow-py3/2313102/testr_results.html.gz | 16:47 |
| *** raildo has quit IRC | 16:48 | |
| ralonsoh | in both examples, one VM obtains IP and not the other one | 16:48 |
| slaweq | ralonsoh: in both cases it was IMO the same vm | 16:48 |
| *** lpetrut has quit IRC | 16:49 | |
| slaweq | but it was resizes/cold migrated | 16:49 |
| slaweq | so initially it got IP | 16:49 |
| ralonsoh | hmmm correct | 16:49 |
| slaweq | and after resize/migration it failed | 16:49 |
| slaweq | that's how I see it | 16:49 |
| slaweq | but I just checked it briefly for now | 16:49 |
| slaweq | but I think that there is potentially some problem (race) there and we should check that | 16:50 |
| slaweq | is there any volunteer maybe? | 16:50 |
| *** ykatabam has quit IRC | 16:51 | |
| ralonsoh | (distracted whistling) | 16:51 |
| bcafarel | (looking intently at the ceiling) | 16:51 |
| slaweq | ok, I will open LP for that and we will see | 16:51 |
| njohnston | (wanders away aimlessly) | 16:51 |
| slaweq | LOL | 16:52 |
| slaweq | You guys rock in such moments :P | 16:52 |
| njohnston | we're quite a bunch, aren't we | 16:52 |
| ralonsoh | hehehehe | 16:52 |
| slaweq | njohnston: yeah, indeed :P | 16:52 |
| bcafarel | that is some team spirit at least :) | 16:53 |
| slaweq | #action slaweq to report LP about connectivity issues after resize/migration | 16:53 |
| slaweq | bcafarel: yeah | 16:53 |
| slaweq | 👍 | 16:54 |
| slaweq | ok, now some good news | 16:54 |
| slaweq | mlavalle recently finally found root cause and fixed issue with router migrations | 16:54 |
| slaweq | patch is merged https://review.opendev.org/#/c/691498/ | 16:54 |
| njohnston | \o/ | 16:55 |
| ralonsoh | good one | 16:55 |
| slaweq | so I hope we should have better numbers on dvr multinode scenario job now | 16:55 |
| slaweq | ralonsoh: yes, that was interesting issue | 16:55 |
| slaweq | in fact it was bug in our code, not in tests | 16:55 |
| slaweq | ok, and the last one from me | 16:56 |
| slaweq | related more to grenade jobs | 16:56 |
| slaweq | I saw that some jobs are failing due to issue in nova scheduler | 16:56 |
| slaweq | so I opened bug https://bugs.launchpad.net/nova/+bug/1850291 | 16:56 |
| openstack | Launchpad bug 1844929 in OpenStack Compute (nova) "duplicate for #1850291 grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] | 16:56 |
| * bcafarel waits for the https://review.opendev.org/#/c/691498/ backports to appear | 16:57 | |
| slaweq | but mriedem just marked it as duplicate of some other one, so it is already known issue for nova team | 16:57 |
| slaweq | and that's all on my side for today | 16:57 |
| ralonsoh | just in time | 16:58 |
| njohnston | I have one thing. Looking at the numbers for the *-uwsgi jobs, they closely mirror their non-uwsgi versions but often are a bit lower. I'd like to propose we make them voting. I'll propose a change and we can conduct the conversation in Gerrit. | 16:58 |
| slaweq | next week I will cancel ci meeting as we will be on ptg | 16:58 |
| ralonsoh | njohnston, are you sure? | 16:58 |
| ralonsoh | ok, let's talk in gerrit | 16:58 |
| slaweq | njohnston++ | 16:59 |
| njohnston | ralonsoh: I set up some custom graphite searches and it looked pretty good to me. The tempest uwsgi job has veen quite stable since the fix for it went in in mid August | 16:59 |
| *** ekcs has quit IRC | 16:59 | |
| ralonsoh | then perfect for me | 16:59 |
| bcafarel | nice | 16:59 |
| slaweq | njohnston: I remember that there was quite many timeout on this job some time ago | 16:59 |
| *** kopecmartin is now known as kopecmartin|off | 16:59 | |
| *** nanzha has joined #openstack-meeting | 16:59 | |
| slaweq | but I don't know exactly how it is now | 16:59 |
| slaweq | lets talk about it on gerrit | 16:59 |
| slaweq | ok, thx for attending | 16:59 |
| njohnston | it's much better, as you can see in the current Grafana - the uwsgi lines are low and tight | 16:59 |
| njohnston | thanks all | 16:59 |
| slaweq | and have a great week | 16:59 |
| njohnston | safe travels all | 17:00 |
| slaweq | o/ | 17:00 |
| ralonsoh | bye! | 17:00 |
| bcafarel | o/ | 17:00 |
| njohnston | o/ | 17:00 |
| slaweq | #endmeeting | 17:00 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 17:00 | |
| openstack | Meeting ended Tue Oct 29 17:00:03 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-10-29-16.00.html | 17:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-10-29-16.00.txt | 17:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-10-29-16.00.log.html | 17:00 |
| *** jamesmcarthur_ has quit IRC | 17:00 | |
| mriedem | slaweq: yup, known for awhile, and i have no great ideas on how to fix it | 17:01 |
| mriedem | nor does anyone else it seems | 17:01 |
| slaweq | mriedem: thx for checking it, and sorry for opening duplicate | 17:01 |
| *** ekcs has joined #openstack-meeting | 17:02 | |
| *** yamamoto has joined #openstack-meeting | 17:02 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:03 | |
| *** jamesmcarthur has quit IRC | 17:03 | |
| mriedem | not a problem | 17:03 |
| *** jamesmcarthur has joined #openstack-meeting | 17:03 | |
| *** nanzha has quit IRC | 17:05 | |
| *** ekcs has quit IRC | 17:05 | |
| *** ekcs has joined #openstack-meeting | 17:08 | |
| *** igordc has joined #openstack-meeting | 17:16 | |
| *** rpittau is now known as rpittau|afk | 17:25 | |
| *** jawad_axd has joined #openstack-meeting | 17:26 | |
| *** gmann is now known as gmann_afk | 17:29 | |
| *** yamamoto has quit IRC | 17:32 | |
| *** gmann_afk is now known as gmann | 17:35 | |
| *** davidsha has quit IRC | 17:37 | |
| *** raildo has joined #openstack-meeting | 17:39 | |
| *** ociuhandu has joined #openstack-meeting | 17:40 | |
| *** ociuhandu has quit IRC | 17:45 | |
| *** jamesmcarthur has quit IRC | 17:47 | |
| *** jamesmcarthur_ has joined #openstack-meeting | 17:47 | |
| *** rsimai has quit IRC | 18:03 | |
| *** rsimai has joined #openstack-meeting | 18:03 | |
| *** ociuhandu has joined #openstack-meeting | 18:12 | |
| *** ociuhandu has quit IRC | 18:18 | |
| *** senrique_ has joined #openstack-meeting | 18:40 | |
| *** enriquetaso has quit IRC | 18:42 | |
| *** ekcs has quit IRC | 18:50 | |
| *** ekcs has joined #openstack-meeting | 18:52 | |
| corvus | o/ | 18:59 |
| clarkb | hello | 18:59 |
| clarkb | we will get started shortly | 18:59 |
| ianw | o/ | 19:00 |
| *** Shrews has joined #openstack-meeting | 19:00 | |
| clarkb | #startmeeting infra | 19:01 |
| openstack | Meeting started Tue Oct 29 19:01:16 2019 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 19:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 19:01 |
| *** openstack changes topic to " (Meeting topic: infra)" | 19:01 | |
| openstack | The meeting name has been set to 'infra' | 19:01 |
| clarkb | #link http://lists.openstack.org/pipermail/openstack-infra/2019-October/006502.html Our Agenda | 19:01 |
| clarkb | #topic Announcements | 19:01 |
| *** openstack changes topic to "Announcements (Meeting topic: infra)" | 19:01 | |
| clarkb | The summit and ptg are next week in shanghai, china | 19:02 |
| clarkb | For this reason we'll cancel next week's meeting | 19:02 |
| Shrews | safe travels to all going | 19:03 |
| fungi | thanks! | 19:04 |
| clarkb | I expect we'll meet per usual the week after | 19:04 |
| clarkb | I'll be jet lagged but around | 19:04 |
| fungi | same | 19:05 |
| corvus | i'll be away that week | 19:05 |
| clarkb | I expect it will be quiet all around due to everyone else traveling/vacationing/being jetlagged | 19:06 |
| clarkb | but I'll do my best to recap the events in shanghai that tuesday | 19:07 |
| clarkb | #topic Actions from last meeting | 19:07 |
| *** openstack changes topic to "Actions from last meeting (Meeting topic: infra)" | 19:07 | |
| clarkb | #link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-10-22-19.01.txt minutes from last meeting | 19:07 |
| clarkb | No actions | 19:08 |
| clarkb | #topic Specs approval | 19:08 |
| *** openstack changes topic to "Specs approval (Meeting topic: infra)" | 19:08 | |
| *** Guest85345 is now known as benj_ | 19:08 | |
| clarkb | #link https://review.opendev.org/#/c/683852/ Migrating off of static.openstack.org spec | 19:08 |
| * fungi saw a spec approved | 19:08 | |
| *** ralonsoh has quit IRC | 19:08 | |
| clarkb | well I haven't approved it yet because fungi was the only person to review its latest patchset | 19:08 |
| clarkb | current plan is to review it today giving everyone a last chance to look it over | 19:09 |
| clarkb | corvus: ^ did you want to check that the edits cover your comment? | 19:09 |
| fungi | oh! i misread the gerrit notification in that case | 19:09 |
| corvus | will do | 19:09 |
| clarkb | thank you! | 19:09 |
| fungi | but i suppose i should still be thrilled that i managed to review something for a change | 19:09 |
| *** AJaeger has joined #openstack-meeting | 19:09 | |
| clarkb | #topic Priority Efforts | 19:09 |
| *** openstack changes topic to "Priority Efforts (Meeting topic: infra)" | 19:09 | |
| clarkb | #topic OpenDev | 19:10 |
| *** openstack changes topic to "OpenDev (Meeting topic: infra)" | 19:10 | |
| clarkb | #link https://etherpad.openstack.org/p/rCF58JvzbF Governance email draft | 19:10 |
| AJaeger | sorry, late joining due to day light saving time switch ;( | 19:10 |
| fungi | did it actually save you any daylight? | 19:10 |
| clarkb | Speaking of things to review ^ if you have thoughts or comments on that they would be much appreciated. At this rate I rpobably won't send it until after the summit anyway | 19:10 |
| fungi | i'm good with it, only left the one comment | 19:10 |
| fungi | which was more a question for the group i suppose | 19:11 |
| * AJaeger is also fine with the email | 19:12 | |
| corvus | clarkb: lgtm | 19:12 |
| clarkb | thanks | 19:13 |
| clarkb | #topic Update Config Management | 19:13 |
| *** openstack changes topic to "Update Config Management (Meeting topic: infra)" | 19:13 | |
| clarkb | I know mordred has been making progress with review-dev recently though I've not been able to keep up the last few days | 19:13 |
| clarkb | he isn't here to correct me but I believe the ansible has been run by hand and is working and now we just need to get changes merged | 19:13 |
| clarkb | #link https://review.opendev.org/#/c/630406/69 stack starts there | 19:14 |
| clarkb | if you have time to review I'm sure mordred would be happy for that and we can probably let mordred approve things that he is confident in? though it is also the -dev server so not really scary in any case | 19:14 |
| clarkb | Any other config management related items to bring up? | 19:15 |
| clarkb | #topic Storyboard | 19:17 |
| *** openstack changes topic to "Storyboard (Meeting topic: infra)" | 19:17 | |
| fungi | we've got working webclient drafts again! | 19:18 |
| clarkb | via the plan to wildcard sources? | 19:18 |
| fungi | all it took was a couple new config features for the api server and supporting an additional action for the auth endpoint | 19:19 |
| fungi | yeah, now cors and openid acls can have entries which start with ^ and those are treated as a regex | 19:19 |
| diablo_rojo | \o/ | 19:20 |
| fungi | however we discovered that the client redirect url is so long on our draft builds (due to object storage) that launchpad switches from get to post for passing the data along to the api | 19:20 |
| fungi | so that endpoint needed to be adjusted to also accept post | 19:20 |
| fungi | and then of course we updated the config for storyboard-dev to basically allow webclients hosted anywhere | 19:21 |
| clarkb | because our rax swift cdn hosted files can come from ~3 * 4096 urls? | 19:22 |
| fungi | well, that's just the hostnames | 19:22 |
| fungi | but yeah | 19:22 |
| fungi | previously storyboard-dev.o.o allowed clients at storyboard-dev.o.o and logs.o.o | 19:23 |
| clarkb | anything else on this subject or should we continue? | 19:24 |
| fungi | there's been some new changes pushed by SotK to support deploying on newer platforms | 19:25 |
| fungi | but no, nothing else i'm aware of to highlight right now | 19:25 |
| clarkb | #topic General topics | 19:26 |
| *** openstack changes topic to "General topics (Meeting topic: infra)" | 19:26 | |
| clarkb | First up is graphite.opendev.org | 19:26 |
| clarkb | We've seen a few questions come up about this so I wanted to make sure everyone was up to speed on the situation there. | 19:26 |
| clarkb | Sometime last week we ran out of disk on the server. After investigating I found taht the stats.timers. stats paths in particular were using the bulk of the disk | 19:26 |
| clarkb | From there we updated the config to reduce retention of that subpath to a year (instead of 5 years) as well as reducing the granularity in that time period | 19:27 |
| clarkb | in order for that to take effect we had to stop services and run whisper-resize.py across all of the files which took ~3 days | 19:27 |
| clarkb | This means that we have a statsd gap of ~3 days, but all the data should match its config and we have plenty of disk to spare too | 19:28 |
| corvus | in retrospect, we probably didn't need to change the retention period, as it's likely that the openstacksdk bug was responsible for a large amount of the space used. | 19:28 |
| clarkb | corvus: ya the sdk bug was a good chunk of it | 19:29 |
| corvus | i confess i was partial to the old retention period -- i've found year-on-year stats to be really helpful in the past | 19:29 |
| clarkb | corvus: I expect we can increase the retention now without doing a whisper resize | 19:29 |
| clarkb | since the resize is for reducing retention? | 19:29 |
| *** eharney has quit IRC | 19:29 | |
| fungi | yeah, though we've lost the old stats now presumably | 19:30 |
| clarkb | but before we do that we should check that sdk hasn't created a bunch of resource id specific files | 19:30 |
| corvus | i'm not entirely sure about that; i thought the retention was encoded in the whisper files when they were created? | 19:30 |
| Shrews | was this the sdk bug that was already fixed? | 19:30 |
| clarkb | fungi: yes corvus ah | 19:30 |
| corvus | Shrews: yep | 19:30 |
| clarkb | Shrews: we think it may have been fixed and cleared out the data that was buggy so that it iwll be apparent if it returns | 19:30 |
| Shrews | *nod* | 19:30 |
| clarkb | Shrews: I can check the filesystem after the meeting to see if new buggy resources have shown up | 19:30 |
| clarkb | note the swift api is still "buggy" | 19:30 |
| clarkb | but that is because its api is different | 19:31 |
| *** yamamoto has joined #openstack-meeting | 19:31 | |
| clarkb | and for our use case has a tiny impact comparatively | 19:31 |
| corvus | clarkb: how is that manifesting? | 19:31 |
| clarkb | corvus: containers and object names get stats files | 19:31 |
| corvus | clarkb: we make a lot of objects | 19:31 |
| clarkb | its probably ok to have per container stats but per objcet is a bit much | 19:31 |
| clarkb | disk consumption is like 70MB for swift vs 200GB for nova though | 19:32 |
| clarkb | so I think we can sort that out without it being an emergency | 19:32 |
| corvus | why is nova so large? | 19:32 |
| clarkb | corvus: I think it is because we only report swift stats to statsd for image uploads to rax. But compute was recording every instance as its own thing | 19:33 |
| *** senrique__ has joined #openstack-meeting | 19:33 | |
| *** ekcs has quit IRC | 19:33 | |
| clarkb | we create ~20 images a day and ~20000 instances | 19:33 |
| corvus | oh, 200 is the old nova usage | 19:33 |
| clarkb | ya | 19:33 |
| fungi | verb tenses to the rescue | 19:34 |
| corvus | i am understanding better now than i was before | 19:34 |
| clarkb | The other thing we will want to bring up with sdk is that splitting operations by tenant or cloud is likely desireable | 19:34 |
| corvus | it doesn't? | 19:35 |
| clarkb | no project ids are removed from the urls (or that is the intent and was failing with the bug) and I don't think there is any cloud info | 19:35 |
| clarkb | this means our statsd data isn't useful since we have a bunch of clouds all reporting to the same buckets and it will end up being noise | 19:35 |
| corvus | Shrews: can nodepool pass in a statsd prefix with the provider info? | 19:35 |
| *** senrique_ has quit IRC | 19:36 | |
| corvus | clarkb: yes, that means it's completely useless for the purpose we created it for | 19:36 |
| Shrews | corvus: we can probably make it so | 19:36 |
| corvus | but iirc, i think maybe the statsd prefix is something that can be passed in per-connection... | 19:36 |
| clarkb | corvus: ah if so then we'll need to update the config but that is doable | 19:36 |
| corvus | if so, then nodepool could fix that for us without a change to sdk, since nodepool provider is how we want that grouped anyway | 19:36 |
| clarkb | ++ | 19:37 |
| corvus | for reference, these starts are supposed to enable the graphs at the bottom of pages like http://grafana.openstack.org/d/BhcSH5Iiz/nodepool-ovh?orgId=1 | 19:37 |
| corvus | under "api operations" | 19:37 |
| corvus | s/starts/stats/ | 19:38 |
| clarkb | we can follow up on that after the meeting. Basically there may be other sdk bugs or nodepool bugs and we should fix them but the emergency is over | 19:38 |
| clarkb | Next up is PTG/Summit planning | 19:39 |
| clarkb | I mentioned this last week but for those of us attending we should expect to be flexible and roll with things that pop up | 19:39 |
| clarkb | #link https://etherpad.openstack.org/p/OpenDev-Shanghai-PTG-2019 Feel free to add topics here for discussion/ptg hacking | 19:40 |
| fungi | i finally got my entry visa yesterday, so i am definitely going to try to be there (barring issues with any of the various modes of transportation which lurk between here and there) | 19:40 |
| clarkb | #link https://www.openstack.org/ptg/#tab_schedule PTG Schedule | 19:40 |
| clarkb | Monday night during the booth crawl thing is a meet the project leaders time as well as lunch on wednesday | 19:40 |
| clarkb | I plan to be there wearning my opendev and zuul hats | 19:41 |
| clarkb | other than that I expect it will be an adventure | 19:41 |
| * fungi needs a zuul logo fez | 19:41 | |
| clarkb | Next I wanted to bring up the opensuse-150 image cleanup | 19:42 |
| clarkb | we've been cleaning up images recently (opensuse-423, fedora-28) | 19:42 |
| clarkb | opensuse 150 is next on the cleanup list (was replaced by opensuse-15) | 19:43 |
| clarkb | #link https://review.opendev.org/#/q/topic:use-opensuse-15 | 19:43 |
| AJaeger | clarkb: I'm using that topic now as well... | 19:43 |
| clarkb | cleaning these up gives us capacity to add newer images and reduces overall load (building and uploading large images is a time sink) | 19:43 |
| clarkb | AJaeger: thank you | 19:43 |
| AJaeger | (but not for those in flight) | 19:43 |
| clarkb | Also be aware that by removing fedora-28 we may break jobs that were consuming those atomic images | 19:44 |
| clarkb | I think the official story from us is use 29 or 30 isntead | 19:44 |
| clarkb | and finally fungi any updates on the wiki replacement? | 19:45 |
| ianw | (that should all be cleaned up & released as of now) | 19:45 |
| fungi | i've not found time to press further on the wiki. looks like i need to start checking for errors logged to apache as the next step | 19:45 |
| fungi | likely it will entail sync'ing up the main config for it | 19:46 |
| fungi | i've still got a stack of changes for it which need review too | 19:46 |
| fungi | topic:xenial-upgrades project:opendev/puppet-mediawiki | 19:47 |
| fungi | getting those landed will help me make more progress | 19:47 |
| * clarkb adds them to the review list | 19:47 | |
| clarkb | #topic Open Discussion | 19:48 |
| *** openstack changes topic to "Open Discussion (Meeting topic: infra)" | 19:48 | |
| clarkb | I am likely to take it easy the next couple days as I've traveling straight through the weekend and want a day or two of rest prior to the summit marathon | 19:48 |
| fungi | #link https://review.opendev.org/691939 Replace old Train cycle signing key with Ussuri | 19:48 |
| corvus | i leave tomorrow and don't expect to be around the rest of this week | 19:49 |
| fungi | if any config-core reviewers is up for approving that today then we don't need to amend the dependent change for the release docs | 19:49 |
| clarkb | +2 from me | 19:49 |
| ianw | #link https://storyboard.openstack.org/#!/story/2006762 | 19:50 |
| ianw | this is a story to replace our mirrors with opendev.org versions | 19:50 |
| *** jamesmcarthur_ has quit IRC | 19:50 | |
| clarkb | ianw: thanks for putting that together I guess we should all grab one or two and start getting them done | 19:51 |
| fungi | i drive to durham thursday and then fly out friday morning. need to find out if Shrews has any ideas for what i should be doing around there on halloween | 19:51 |
| ianw | yeah, if anyone wants to do mirrors, welcome ... | 19:51 |
| ianw | and speaking off, i've spent an increasing amount of time dealing with broken vos releases lately | 19:52 |
| ianw | #link https://review.opendev.org/691824 | 19:52 |
| ianw | there's hopefully sufficient detail in that review, but it has been suggested that using -localauth with "vos release" is really the best you can do | 19:52 |
| ianw | if that idea is acceptable to us, i'd like to get that in and probably stage it with fedora initially | 19:53 |
| clarkb | in that case I guess we rely on ssh to be a reliable connection but it typically is so should be fine | 19:54 |
| fungi | did you ever manage to work out why vos release of the fedora volumes takes so very, very long to complete? | 19:54 |
| ianw | sort of, not really ... | 19:55 |
| *** panda|pto has quit IRC | 19:55 | |
| ianw | https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/mirror-update/files/fedora-mirror-update#L124 | 19:55 |
| ianw | is about the best guess ... but it still seems to want to do full releases constantly | 19:55 |
| corvus | that is not how i understood vos release to work, but i'm no expert | 19:57 |
| fungi | ouch, that's quite the strangeness | 19:58 |
| *** panda has joined #openstack-meeting | 19:58 | |
| clarkb | We are just about at time. Thank you everyone ! We'll be back in 2 weeks. | 19:58 |
| corvus | i thought it behaved more like your standard cow | 19:58 |
| corvus | but i guess there's lots of varieties of cows | 19:59 |
| clarkb | Feel free to continue discussions over in #openstack-infra or on the mailing list but I'm going to formally end the meeting now | 19:59 |
| clarkb | #endmeeting | 19:59 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 19:59 | |
| openstack | Meeting ended Tue Oct 29 19:59:15 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 19:59 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-10-29-19.01.html | 19:59 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-10-29-19.01.txt | 19:59 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-10-29-19.01.log.html | 19:59 |
| fungi | thanks clarkb! | 19:59 |
| *** AJaeger has left #openstack-meeting | 19:59 | |
| *** Shrews has left #openstack-meeting | 20:00 | |
| *** rsimai has quit IRC | 20:03 | |
| *** rsimai has joined #openstack-meeting | 20:04 | |
| *** jamesmcarthur has joined #openstack-meeting | 20:09 | |
| *** mordred has joined #openstack-meeting | 20:24 | |
| *** eharney has joined #openstack-meeting | 20:27 | |
| *** markvoelker has quit IRC | 20:35 | |
| *** markvoelker has joined #openstack-meeting | 20:45 | |
| *** rbudden has joined #openstack-meeting | 20:50 | |
| *** trandles has joined #openstack-meeting | 20:56 | |
| *** oneswig has joined #openstack-meeting | 20:57 | |
| *** ekcs has joined #openstack-meeting | 20:59 | |
| *** jamesmcarthur has quit IRC | 20:59 | |
| oneswig | #startmeeting scientific-sig | 21:00 |
| openstack | Meeting started Tue Oct 29 21:00:15 2019 UTC and is due to finish in 60 minutes. The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:00 |
| *** openstack changes topic to " (Meeting topic: scientific-sig)" | 21:00 | |
| openstack | The meeting name has been set to 'scientific_sig' | 21:00 |
| oneswig | ahoy! | 21:00 |
| trandles | hello | 21:00 |
| oneswig | #link agenda for today https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_October_29th_2019 | 21:00 |
| oneswig | Hi trandles, afternoon | 21:00 |
| oneswig | What's new? | 21:01 |
| trandles | Nothing much, just super busy | 21:01 |
| trandles | You preparing for Shanghai? | 21:01 |
| oneswig | I have a visa in my passport, at least | 21:02 |
| oneswig | Just looking through the schedule at the moment... | 21:02 |
| trandles | I haven't had a chance to see the schedule. Mostly been focusing on SC | 21:03 |
| oneswig | Some useful talks about operations at scale. Definitely on my radar. | 21:03 |
| *** b1airo has joined #openstack-meeting | 21:03 | |
| oneswig | trandles: are you presenting material at SC? | 21:03 |
| *** janders has joined #openstack-meeting | 21:04 | |
| trandles | I'm a co-author on 3 posters (one for CANOPIE) but not presenting | 21:04 |
| trandles | Well, the CANOPIE submission is a poster and a short paper | 21:04 |
| janders | g'day all! | 21:04 |
| trandles | Hi janders | 21:04 |
| oneswig | hi janders | 21:04 |
| b1airo | o/ | 21:05 |
| oneswig | Hi b1airo! | 21:05 |
| oneswig | #chair b1airo | 21:05 |
| openstack | Current chairs: b1airo oneswig | 21:05 |
| oneswig | How's everyone enjoying the rugby? :-) | 21:05 |
| b1airo | that's it, i'm out! | 21:05 |
| b1airo | such a horrible game from ABs | 21:06 |
| oneswig | got to make hay when the sun shines, particularly with the all-blacks... | 21:06 |
| trandles | oneswig gets a +1 for htat | 21:06 |
| trandles | *that | 21:06 |
| b1airo | indeed, they couldn't get their act together after realising England were there to play and play hard | 21:07 |
| oneswig | With a 9am kick-off back home, many people lost a weekend after that... | 21:07 |
| b1airo | not sure who to barrack for in the final | 21:08 |
| oneswig | can't remember - who's in it b1airo? | 21:08 |
| oneswig | :-) | 21:08 |
| trandles | lol | 21:08 |
| trandles | relentless | 21:08 |
| oneswig | I'm sure we had an agenda somewher | 21:08 |
| b1airo | All Blacks are just helping prepare the nation for the cricket season... | 21:09 |
| oneswig | ah, at least you have summer to look forwards to | 21:10 |
| oneswig | Anyone going to Shanghai from NESI? | 21:10 |
| b1airo | no, not this time round i'm afraid. there was some interest from NIWA but couldn't get the travel sorted/approved | 21:11 |
| oneswig | too bad. | 21:11 |
| oneswig | I'm interested in the talk from Pawsey | 21:12 |
| *** rfolco|rover is now known as rfolco|off | 21:12 | |
| oneswig | ... which doesn't appear to be in the schedule any more... | 21:13 |
| b1airo | oh, what was it about? their not Nectar Nectar-cloud ... :-) | 21:14 |
| *** martial has joined #openstack-meeting | 21:14 | |
| oneswig | Using CephFS as a cluster filesystem for MPI workloads, IIRC. | 21:14 |
| oneswig | My personal experience was mostly in the "not yet" camp | 21:15 |
| *** pcaruana has quit IRC | 21:15 | |
| oneswig | But we have some good operational experience of it serving home dirs for non-MPI workloads. | 21:15 |
| janders | look like it's been removed indeed :( | 21:16 |
| b1airo | oh ok, i didn't know they had been attempting anything like that. I know UniMelb have been, with fairly middling results. i would agree with the "not yet" assessment | 21:16 |
| janders | I'd love to watch it | 21:16 |
| b1airo | in fact, i'm kind of skeptical about it ever being a good option for that | 21:16 |
| janders | yeah that's why we went GPFS instead (which was running fine till the IB switch died, it's having a break while RMA is being sorted) | 21:17 |
| oneswig | Maybe they had second thoughts about their experiences? | 21:17 |
| janders | it hasn't been a good month for storage for ue | 21:17 |
| janders | *us | 21:17 |
| b1airo | i can see it being useful for all persistent storage applications in HPC, all the way to "project/programme" space, but for true scratch it's just not a good design fit | 21:17 |
| oneswig | It's not anything like as fast. We've used it for hybrid cloud. | 21:18 |
| janders | I was quite disappointed yesterday to hear 1) BeeGFS doesn't have any QoS and 2) it's not on the roadmap | 21:19 |
| janders | I suppose BeeOND can cover that off in a way but still | 21:19 |
| oneswig | janders: what's happened this month? | 21:20 |
| janders | do you guys have any experience where BeeOND for job X is running on nodes where job Y is running? | 21:20 |
| janders | oneswig: where do I start... | 21:20 |
| janders | we had some metadata lockups on BeeGFS not handled by builtin HA which caused a lot of damage | 21:21 |
| oneswig | No, sorry, sounds like a proper noisy neighbour issue | 21:21 |
| janders | and last week HDR switch died in my OpenStack kit | 21:21 |
| oneswig | janders: data loss? | 21:21 |
| janders | not much of that but a lot of downtime at the least appropriate time | 21:22 |
| oneswig | HDR switch, but that's practically brand new! | 21:22 |
| janders | yeah hence RMAs come out of Israel | 21:22 |
| b1airo | the joys of Mellanox support | 21:22 |
| janders | no local stock | 21:22 |
| oneswig | On a related note, these guys didn't get the news: https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24298/hpc-and-openstack-via-omni-path-architecture | 21:23 |
| janders | I designed the system with one IB switch as it's just under 70 nodes (and I would struggle to fit in the budget otherwise) | 21:23 |
| oneswig | "future work"... | 21:23 |
| janders | haha | 21:23 |
| oneswig | 70 x 100G or 70x 200G? | 21:23 |
| janders | but I'll see if I can get more switching there - it would be nice to experiment with IB-multipath-GPFS | 21:23 |
| janders | a combination | 21:24 |
| janders | 80%-20% HDR100-HDR200 | 21:24 |
| janders | GPFS and some clients are 200 | 21:24 |
| b1airo | lol! seriously though, how did that talk get in..? | 21:25 |
| janders | through side avenues? :) | 21:25 |
| janders | or maybe keyword sponsored? | 21:25 |
| b1airo | maybe it replaced the Pawsey one | 21:25 |
| oneswig | I'm sure it has nothing to do with affiliation with a Summit Diamond Sponsor | 21:26 |
| janders | as much as I dont think this is a relevant direction it would be interesting to have a look at the mechanics of the integration mechanism | 21:27 |
| oneswig | Ironic without PXE seems interesting https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24274/ironically-reeling-in-large-scale-bare-metal-deployment-without-pxe | 21:27 |
| janders | cause I asked Intel exactly that a year and a bit back and they were like "uh... oh... we don't think we can do that." | 21:27 |
| b1airo | anyone here on the selection panel? my experience is the panel makes the final decision, have never seen it changed by foundation staff except for logistical reasons | 21:27 |
| oneswig | janders: right, and there's a few OPA systems out there. | 21:27 |
| janders | that PXEless talk is cool | 21:28 |
| janders | Julia mentioned some of the underlying bits at the PTG in Denver | 21:28 |
| oneswig | b1airo: think you're right. I saw the panel members at one point. | 21:28 |
| janders | looks like it's coming together | 21:28 |
| b1airo | hasn't the Dell Ironic driver always worked without PXE...? | 21:29 |
| oneswig | The iDrac driver and "always worked" in the same sentence, hmmm. | 21:30 |
| janders | LOL and +1 | 21:30 |
| janders | I used Ironic with Dell kit a fair bit, but it was always generic driver | 21:30 |
| oneswig | We've spent some time with it again recently, still hit issues with stuck jobs | 21:30 |
| janders | mostly due to the fact no one I spoke to at Dell was able to explain how to use it | 21:30 |
| janders | Arkady stepped in to help some time back but I moved to Intel kit since | 21:31 |
| janders | I think at least the doco should be less bad now | 21:31 |
| *** rfolco|off has quit IRC | 21:31 | |
| oneswig | I recall there were some useful-looking Ansible playbooks built upon it in the last couple of years | 21:32 |
| *** yamamoto has quit IRC | 21:33 | |
| janders | did it get a bit quiet, or did I drop out? | 21:37 |
| *** senrique__ has quit IRC | 21:37 | |
| b1airo | oneswig: pretty sure i saw those playbooks in a blog of yours... | 21:38 |
| oneswig | Not sure about that exactly, there were others we never got into using | 21:40 |
| oneswig | Don't recall the names, alas | 21:40 |
| b1airo | for BM clouds it does kind of feel like we need a better way of managing server imaging... there must be a way to pre-deploy common OS images ready for boot selection. some sort of boot drive mechanism that let's you deploy images and select the desired image in management mode, then in runtime mode it COW overlays the selected OS image | 21:41 |
| *** ykatabam has joined #openstack-meeting | 21:43 | |
| oneswig | It does seem quite clunky, agreed. | 21:43 |
| oneswig | I recall there was a switch a few years back from a company called Pluribus Networks that was all about pushing compute and data services out to the ToR | 21:44 |
| *** bbowen has quit IRC | 21:45 | |
| oneswig | fancy that ... looks like they are still at it. | 21:45 |
| b1airo | yeah, i seem to recall trandles dreaming up a ToR/rack-oriented architecture like this too | 21:47 |
| oneswig | far out | 21:47 |
| trandles | ah yeah, scalability in provisioning | 21:47 |
| trandles | Our work is still in progress | 21:48 |
| oneswig | Has your new team member started trandles? | 21:49 |
| trandles | b1airo, what you're describing in the "BM clouds" comment is kinda like what NERSC did with CHOS | 21:49 |
| trandles | But CHOS was kinda clunky and used chroot | 21:49 |
| trandles | oneswig, negative...but should be by the end of the year | 21:49 |
| trandles | https://github.com/NERSC/chos | 21:51 |
| oneswig | in a chroot, can you still mknod a device file for the raw root device and mount it? | 21:52 |
| *** slaweq has quit IRC | 21:52 | |
| oneswig | asking for a friend :-) | 21:53 |
| b1airo | ah yeah, i think i've seen / heard of that before. i guess to do this properly(tm) would require some sort of integration with UEFI boot etc to avoid a guest OS instance from fiddling with the pre-deployed images | 21:54 |
| oneswig | ... or could the CH be CHarliecloud? | 21:54 |
| trandles | I'm much more in favor of something like kexec + pivot_root for rapid re-provisioning | 21:55 |
| trandles | kexec (if a new kernel is required) and then the pivot_root to the new image...Ironic has talked about supporting it | 21:56 |
| trandles | I had a phone call with some people a couple months ago who are interested | 21:56 |
| oneswig | In an Ironic context? | 21:56 |
| trandles | yeah | 21:57 |
| oneswig | It does violate that abstraction of the guest OS, but perhaps we just want faster boots, man! | 21:57 |
| b1airo | yeah that sounds fine for a trusted environment / set of tenants, but in a true multi-tenant cloud env you'd need a method to protect (or at least verify) the OS image/install is good | 21:58 |
| oneswig | trandles: I still haven't tried the ramdisk deploy driver. Been meaning to for ages. | 21:58 |
| trandles | Either let us replace UEFI with a kernel and ramdisk, or give us ways to "reboot" without going back through the firmware please | 21:58 |
| trandles | oneswig: I had half a day a month ago to play with it some, but managed to mess something up along the way and haven't gone back to it yet :( | 21:59 |
| oneswig | ... or give us a little game to play while the lifecycle controller does its stuff in the background? | 21:59 |
| oneswig | ah, we are at time. | 21:59 |
| oneswig | Any more? | 21:59 |
| b1airo | oh, whoops - i meant to leave for a meeting 10 minutes ago! | 22:00 |
| b1airo | cya! | 22:00 |
| oneswig | OK, nice talking to you again guys. I'll put some of these talking points on the flip chart for the SIG PTG. | 22:00 |
| janders | "standby" mode for ironic was on the radar for the current Ironic cycle | 22:00 |
| oneswig | See you in Denver! | 22:00 |
| oneswig | #endmeeting | 22:00 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 22:00 | |
| openstack | Meeting ended Tue Oct 29 22:00:45 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 22:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-10-29-21.00.html | 22:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-10-29-21.00.txt | 22:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-10-29-21.00.log.html | 22:00 |
| janders | thanks guys! | 22:00 |
| trandles | later everyone | 22:01 |
| janders | (sorry had to pop out) | 22:01 |
| *** trandles has quit IRC | 22:01 | |
| *** janders has quit IRC | 22:01 | |
| *** oneswig has quit IRC | 22:01 | |
| *** rcernin has joined #openstack-meeting | 22:02 | |
| *** mriedem has quit IRC | 22:06 | |
| *** raildo has quit IRC | 22:08 | |
| *** e0ne has joined #openstack-meeting | 22:22 | |
| *** e0ne has quit IRC | 22:26 | |
| *** gyee has quit IRC | 22:27 | |
| *** e0ne has joined #openstack-meeting | 22:28 | |
| *** e0ne has quit IRC | 22:29 | |
| *** e0ne has joined #openstack-meeting | 22:33 | |
| *** e0ne has quit IRC | 22:36 | |
| *** e0ne has joined #openstack-meeting | 22:39 | |
| *** e0ne has quit IRC | 22:42 | |
| *** e0ne has joined #openstack-meeting | 22:45 | |
| *** e0ne has quit IRC | 22:48 | |
| *** slaweq has joined #openstack-meeting | 22:50 | |
| *** e0ne has joined #openstack-meeting | 22:51 | |
| *** ekcs has quit IRC | 22:52 | |
| *** gyee has joined #openstack-meeting | 22:52 | |
| *** ekcs has joined #openstack-meeting | 22:53 | |
| *** slaweq has quit IRC | 22:54 | |
| *** e0ne has quit IRC | 22:56 | |
| *** ykatabam has quit IRC | 23:00 | |
| *** e0ne has joined #openstack-meeting | 23:00 | |
| *** e0ne has quit IRC | 23:03 | |
| *** ekcs has quit IRC | 23:04 | |
| *** e0ne has joined #openstack-meeting | 23:07 | |
| *** ykatabam has joined #openstack-meeting | 23:07 | |
| *** e0ne has quit IRC | 23:10 | |
| *** diablo_rojo has quit IRC | 23:10 | |
| *** diablo_rojo has joined #openstack-meeting | 23:11 | |
| *** e0ne has joined #openstack-meeting | 23:14 | |
| *** e0ne has quit IRC | 23:16 | |
| *** mattw4 has quit IRC | 23:20 | |
| *** senrique__ has joined #openstack-meeting | 23:20 | |
| *** e0ne has joined #openstack-meeting | 23:20 | |
| *** mattw4 has joined #openstack-meeting | 23:20 | |
| *** e0ne has quit IRC | 23:24 | |
| *** mattw4 has quit IRC | 23:27 | |
| *** e0ne has joined #openstack-meeting | 23:28 | |
| *** e0ne has quit IRC | 23:31 | |
| *** yamamoto has joined #openstack-meeting | 23:31 | |
| *** e0ne has joined #openstack-meeting | 23:34 | |
| *** e0ne has quit IRC | 23:37 | |
| *** jamesmcarthur has joined #openstack-meeting | 23:38 | |
| *** e0ne has joined #openstack-meeting | 23:40 | |
| *** e0ne has quit IRC | 23:42 | |
| *** jamesmcarthur has quit IRC | 23:46 | |
| *** brinzhang has joined #openstack-meeting | 23:49 | |
| *** senrique__ has quit IRC | 23:54 | |
| *** gyee has quit IRC | 23:54 | |
| *** yamamoto has quit IRC | 23:57 | |
| *** jamesmcarthur has joined #openstack-meeting | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!