| *** shwetaap has quit IRC | 00:09 | |
| *** markvoelker has quit IRC | 00:26 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 00:26 | |
| *** s3wong has quit IRC | 00:26 | |
| *** s3wong has joined #openstack-meeting-4 | 00:27 | |
| *** eghobo_ has quit IRC | 00:32 | |
| *** carl_baldwin has quit IRC | 00:33 | |
| *** eghobo has joined #openstack-meeting-4 | 00:33 | |
| *** banix has joined #openstack-meeting-4 | 00:34 | |
| *** banix has quit IRC | 00:39 | |
| *** wojdev has quit IRC | 00:43 | |
| *** VW_ has quit IRC | 00:46 | |
| *** Aish has quit IRC | 00:50 | |
| *** mwang2__ has joined #openstack-meeting-4 | 00:51 | |
| *** banix has joined #openstack-meeting-4 | 00:55 | |
| *** mwang2_ has quit IRC | 00:55 | |
| *** madhu has quit IRC | 00:56 | |
| *** eghobo has quit IRC | 00:58 | |
| *** SumitNaiksatam has quit IRC | 01:01 | |
| *** eghobo has joined #openstack-meeting-4 | 01:03 | |
| *** SridharRamaswamy has quit IRC | 01:04 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 01:06 | |
| *** markvoelker has joined #openstack-meeting-4 | 01:09 | |
| *** s3wong_ has joined #openstack-meeting-4 | 01:10 | |
| *** s3wong has quit IRC | 01:12 | |
| *** markvoelker has quit IRC | 01:14 | |
| *** dannywilson has quit IRC | 01:16 | |
| *** salv-orlando has quit IRC | 01:20 | |
| *** madhu has joined #openstack-meeting-4 | 01:26 | |
| *** bobmel has quit IRC | 01:27 | |
| *** bobmel has joined #openstack-meeting-4 | 01:28 | |
| *** eghobo has quit IRC | 01:30 | |
| *** ajmiller has quit IRC | 01:31 | |
| *** eghobo has joined #openstack-meeting-4 | 01:44 | |
| *** sarob has quit IRC | 01:51 | |
| *** banix has quit IRC | 01:53 | |
| *** VW_ has joined #openstack-meeting-4 | 01:53 | |
| *** eghobo has quit IRC | 01:53 | |
| *** VW_ has quit IRC | 02:00 | |
| *** VW_ has joined #openstack-meeting-4 | 02:00 | |
| *** yapeng_ has joined #openstack-meeting-4 | 02:01 | |
| *** ChuckC has quit IRC | 02:04 | |
| *** markvoelker has joined #openstack-meeting-4 | 02:10 | |
| *** Aish has joined #openstack-meeting-4 | 02:12 | |
| *** galstrom_zzz is now known as galstrom | 02:13 | |
| *** markvoelker has quit IRC | 02:15 | |
| *** VW__ has joined #openstack-meeting-4 | 02:16 | |
| *** VW__ has quit IRC | 02:16 | |
| *** VW_ has quit IRC | 02:16 | |
| *** VW_ has joined #openstack-meeting-4 | 02:16 | |
| *** salv-orlando has joined #openstack-meeting-4 | 02:20 | |
| *** markvoelker has joined #openstack-meeting-4 | 02:27 | |
| *** markvoelker has quit IRC | 02:33 | |
| *** s3wong_ has quit IRC | 02:47 | |
| *** banix has joined #openstack-meeting-4 | 02:47 | |
| *** ChuckC has joined #openstack-meeting-4 | 02:49 | |
| *** galstrom is now known as galstrom_zzz | 02:51 | |
| *** VW_ has quit IRC | 02:52 | |
| *** VW_ has joined #openstack-meeting-4 | 02:53 | |
| *** ChuckC has quit IRC | 02:54 | |
| *** ChuckC has joined #openstack-meeting-4 | 02:54 | |
| *** VW_ has quit IRC | 02:56 | |
| *** VW_ has joined #openstack-meeting-4 | 02:57 | |
| *** VW_ has quit IRC | 02:58 | |
| *** VW_ has joined #openstack-meeting-4 | 02:59 | |
| *** VW_ has quit IRC | 02:59 | |
| *** VW_ has joined #openstack-meeting-4 | 03:00 | |
| *** VW_ has quit IRC | 03:00 | |
| *** VW_ has joined #openstack-meeting-4 | 03:00 | |
| *** VW__ has joined #openstack-meeting-4 | 03:04 | |
| *** VW__ has quit IRC | 03:06 | |
| *** ChuckC has quit IRC | 03:06 | |
| *** ChuckC has joined #openstack-meeting-4 | 03:06 | |
| *** VW__ has joined #openstack-meeting-4 | 03:06 | |
| *** VW_ has quit IRC | 03:07 | |
| *** ivar-lazzaro has quit IRC | 03:14 | |
| *** salv-orlando has quit IRC | 03:21 | |
| *** markvoelker has joined #openstack-meeting-4 | 03:28 | |
| *** yamahata has quit IRC | 03:29 | |
| *** markvoelker has quit IRC | 03:33 | |
| *** SumitNaiksatam has joined #openstack-meeting-4 | 03:33 | |
| *** banix has quit IRC | 03:45 | |
| *** ajmiller has joined #openstack-meeting-4 | 04:05 | |
| *** dannywilson has joined #openstack-meeting-4 | 04:13 | |
| *** madhu has quit IRC | 04:16 | |
| *** dannywilson has quit IRC | 04:17 | |
| *** hughsaunders has quit IRC | 04:24 | |
| *** fnaval has quit IRC | 04:26 | |
| *** markvoelker has joined #openstack-meeting-4 | 04:29 | |
| *** shwetaap has joined #openstack-meeting-4 | 04:32 | |
| *** galstrom_zzz is now known as galstrom | 04:32 | |
| *** Aish has left #openstack-meeting-4 | 04:34 | |
| *** markvoelker has quit IRC | 04:34 | |
| *** hughsaunders has joined #openstack-meeting-4 | 04:35 | |
| *** fnaval has joined #openstack-meeting-4 | 04:41 | |
| *** eghobo has joined #openstack-meeting-4 | 05:02 | |
| *** yapeng_ has quit IRC | 05:04 | |
| *** galstrom is now known as galstrom_zzz | 05:07 | |
| *** shwetaap has quit IRC | 05:20 | |
| *** claired has joined #openstack-meeting-4 | 05:22 | |
| *** qwebirc74309 has joined #openstack-meeting-4 | 05:23 | |
| *** claired has quit IRC | 05:25 | |
| *** fnaval has quit IRC | 05:26 | |
| *** markvoelker has joined #openstack-meeting-4 | 05:31 | |
| *** claired has joined #openstack-meeting-4 | 05:31 | |
| *** claired has quit IRC | 05:32 | |
| *** openstack has joined #openstack-meeting-4 | 05:35 | |
| *** ChanServ sets mode: +o openstack | 05:35 | |
| *** markvoelker has quit IRC | 05:36 | |
| *** salv-orlando has joined #openstack-meeting-4 | 05:38 | |
| *** shwetaap has joined #openstack-meeting-4 | 06:03 | |
| *** salv-orlando has quit IRC | 06:29 | |
| *** eghobo has quit IRC | 06:31 | |
| *** markvoelker has joined #openstack-meeting-4 | 06:32 | |
| *** markvoelker has quit IRC | 06:38 | |
| *** wojdev has joined #openstack-meeting-4 | 06:41 | |
| *** wojdev has quit IRC | 06:54 | |
| *** eghobo has joined #openstack-meeting-4 | 06:54 | |
| *** eghobo has quit IRC | 06:55 | |
| *** eghobo has joined #openstack-meeting-4 | 06:56 | |
| *** shwetaap has quit IRC | 07:13 | |
| *** wojdev has joined #openstack-meeting-4 | 07:23 | |
| *** david-lyle_afk has joined #openstack-meeting-4 | 07:33 | |
| *** kobis has joined #openstack-meeting-4 | 07:33 | |
| *** markvoelker has joined #openstack-meeting-4 | 07:34 | |
| *** wojdev has quit IRC | 07:35 | |
| *** ppetit has joined #openstack-meeting-4 | 07:39 | |
| *** markvoelker has quit IRC | 07:39 | |
| *** evgenyf has joined #openstack-meeting-4 | 07:54 | |
| *** eghobo_ has joined #openstack-meeting-4 | 07:57 | |
| *** eghobo has quit IRC | 07:59 | |
| *** eghobo_ has quit IRC | 08:00 | |
| *** salv-orlando has joined #openstack-meeting-4 | 08:15 | |
| *** VW__ has quit IRC | 08:17 | |
| *** yamahata has joined #openstack-meeting-4 | 08:26 | |
| *** matrohon has joined #openstack-meeting-4 | 08:28 | |
| -openstackstatus- NOTICE: Zuul check queue stuck due to reboot maintenance window at one of our cloud providers - no need to recheck changes at the moment, they won't move forward. | 08:30 | |
| *** ChanServ changes topic to "Zuul check queue stuck due to reboot maintenance window at one of our cloud providers - no need to recheck changes at the moment, they won't move forward." | 08:30 | |
| *** markvoelker has joined #openstack-meeting-4 | 08:35 | |
| *** markvoelker has quit IRC | 08:39 | |
| *** salv-orlando has quit IRC | 08:45 | |
| *** salv-orlando has joined #openstack-meeting-4 | 08:45 | |
| *** andymccr has left #openstack-meeting-4 | 08:46 | |
| *** hareeshp has joined #openstack-meeting-4 | 08:50 | |
| *** [1]evgenyf has joined #openstack-meeting-4 | 08:56 | |
| *** evgenyf has quit IRC | 08:59 | |
| *** [1]evgenyf is now known as evgenyf | 08:59 | |
| *** watanabe_isao has quit IRC | 09:02 | |
| *** yapeng_ has joined #openstack-meeting-4 | 09:05 | |
| *** openstack has joined #openstack-meeting-4 | 15:27 | |
| *** ChanServ sets mode: +o openstack | 15:27 | |
| asselin_ | krtaylor, for reviewers, it's easier to use one topic for the 2 specs since they're related. | 15:27 |
|---|---|---|
| *** rhe00 has joined #openstack-meeting-4 | 15:27 | |
| krtaylor | asselin_, yes, I was just referring to "19:27:44 <jeblair> actually the spec said "openstackci" as a topic" | 15:28 |
| krtaylor | whatever, good to clarify here | 15:29 |
| asselin_ | so, we can start working on it now. First we'll create the new repo, and I'll add some tasks to storyboard | 15:29 |
| krtaylor | anyway, what is the plan to move forward, how can we help | 15:29 |
| krtaylor | ok, good | 15:29 |
| asselin_ | ppl can self-assign from there so we don't duplicate effort | 15:30 |
| krtaylor | agreed | 15:30 |
| *** b3rnard0 has left #openstack-meeting-4 | 15:30 | |
| ja_ | do we have a sense of when it would be usable by someone starting from scratch? | 15:30 |
| krtaylor | similar to what we did with puppet-module split out | 15:30 |
| asselin_ | #link https://storyboard.openstack.org/#!/story/2000101 | 15:30 |
| *** ajmiller has joined #openstack-meeting-4 | 15:30 | |
| ja_ | e.g. would we tell someone about to start building a new one to wait 'n' weeks, or proceed now | 15:31 |
| krtaylor | target liberty I would assume? | 15:31 |
| asselin_ | not much yet. I'll populate some items today. Anyone can add stuff too FYI | 15:31 |
| asselin_ | there's no target...it's a priority effort | 15:31 |
| asselin_ | krtaylor, but honestly, it would be great to be done by the liberty summit... | 15:32 |
| krtaylor | well as with puppet-module, use us for help | 15:32 |
| krtaylor | thats aggressive...but maybe doable | 15:32 |
| krtaylor | asselin_, if not a hack day there would be good too | 15:33 |
| asselin_ | krtaylor, yes, we'll probably use that. it was very effective. | 15:33 |
| krtaylor | agreed, the Friday mini sprint is very cool, nice to have everyone in one room | 15:34 |
| asselin_ | after we get a few patches going, we can do a sprint to review and get pieces merged | 15:34 |
| krtaylor | oh, I was talking about at summit | 15:34 |
| krtaylor | but a virtual sprint would be effective too | 15:34 |
| *** evgenyf has quit IRC | 15:35 | |
| asselin_ | yes, maybe after kilo releases during the downtime. in case there are riskier changes | 15:35 |
| krtaylor | agreed, this is the start of really doing something to improve the consumables | 15:37 |
| krtaylor | I encourage everyone to get involved in this | 15:37 |
| krtaylor | ok, anything esle you want to mention asselin_ ? | 15:37 |
| asselin_ | not on this topic | 15:38 |
| krtaylor | ok, onward then | 15:38 |
| krtaylor | #topic Repo for third party tools | 15:39 |
| krtaylor | I have not made any progress on this, this week | 15:39 |
| krtaylor | day job keeps getting in the way | 15:39 |
| krtaylor | but, I plan on returning to this today/tomorrow | 15:40 |
| *** galstrom_zzz is now known as galstrom | 15:40 | |
| krtaylor | here is the etherpad | 15:40 |
| krtaylor | #link https://etherpad.openstack.org/p/third-party-ci-wg-repo | 15:40 |
| krtaylor | there is some good ideas there | 15:40 |
| krtaylor | I am leaning toward starting slower, after thinking about it for a while | 15:40 |
| krtaylor | I think having an index of tools would be a good start | 15:41 |
| krtaylor | then we can see what we have and maybe better organize | 15:41 |
| patrickeast | i like that idea | 15:41 |
| krtaylor | this can also come out of the system highlight discussions | 15:41 |
| krtaylor | also, it gives us a way to collect up some tools and see how many we are talking about | 15:42 |
| krtaylor | if we only have a few that companies are willing to share, then not worth the effort | 15:42 |
| krtaylor | an index into individual github accounts would be great and serve the intent of why we are doing this in the first place | 15:43 |
| krtaylor | please share any thoughts in the etherpad, I will too | 15:44 |
| patrickeast | will do | 15:44 |
| krtaylor | any questions on this? | 15:45 |
| krtaylor | next then | 15:45 |
| krtaylor | #topic What to do with monitoring dashboard | 15:45 |
| krtaylor | is sweston around? | 15:45 |
| krtaylor | there has been some discussion on this | 15:45 |
| krtaylor | I have been thinking about it too | 15:46 |
| krtaylor | it seems that what everyone wanted was to get the previous dashboard (now radar) working again | 15:47 |
| patrickeast | it kind of looks like the spec hasn’t really moved forward in a while | 15:47 |
| krtaylor | yeah | 15:47 |
| patrickeast | oh | 15:47 |
| patrickeast | i showed this at the monday meeting | 15:47 |
| patrickeast | for anyone who missed that one http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack%2Fcinder&user=&timeframe=24 | 15:47 |
| patrickeast | i put together a lame little dash | 15:47 |
| krtaylor | not lame, excellent tool | 15:48 |
| patrickeast | the original intent was that i can’t see when zuul reports a failure (only jenkins) so i wanted to monitor the event stream | 15:48 |
| mmedvede | cool! | 15:48 |
| patrickeast | but i let it watch all the ci accounts | 15:48 |
| patrickeast | still has some serious perf issues | 15:48 |
| patrickeast | needs more than a weekend of abuse to get working | 15:48 |
| krtaylor | yeah, and it just needs some historical stats to be exactly what we need | 15:48 |
| * asselin_ likes it | 15:48 | |
| patrickeast | imo its a different solution than what sweston’s spec was aiming for | 15:49 |
| krtaylor | patrickeast, did you ever see the radar tool working? | 15:49 |
| patrickeast | no control, no stats, trends | 15:49 |
| patrickeast | nope, before my time | 15:49 |
| patrickeast | https://github.com/patrick-east/scoreboard | 15:49 |
| patrickeast | ^ thats the source for it if anyone wants to run it for themselves | 15:49 |
| patrickeast | the ec2 thing that test one is running on sucks | 15:49 |
| patrickeast | works better on a local machine with a little more network bandwidth | 15:50 |
| patrickeast | easily modified to only watch your ci account and jenkins | 15:50 |
| * asselin_ notes his ci needs some work | 15:50 | |
| patrickeast | asselin: hehe its a harsh judge, if 1/5 of your tests fail it marks the ci as a fail | 15:51 |
| krtaylor | radar - it was simple gauges that showed the percentages of passed, failed, skipped tests for each ci system and upstream jenkins | 15:51 |
| patrickeast | ahh gotcha | 15:51 |
| krtaylor | it was a set sample, I believe it was 30 days | 15:51 |
| patrickeast | did it query gerrit or just track events? | 15:52 |
| patrickeast | or both | 15:52 |
| krtaylor | gathered and presented, but the urls changed for CI systems and it broke | 15:52 |
| krtaylor | it was kinda hacky, but worked fine | 15:52 |
| krtaylor | problem is that we don't have a master list of CI systems anymore | 15:53 |
| patrickeast | well, with a little bit of work i can add those kind of stats to this dash if folks are interested | 15:53 |
| krtaylor | except the maillist one | 15:53 |
| patrickeast | i basically record the gerrit events in the db and can do whatever with em | 15:53 |
| lennyb_ | I made a small script to monitor last N Jenkins JObs to see if they failed. can it help you ? | 15:53 |
| krtaylor | jhesketh and I discussed using his gathering backend at paris summit | 15:54 |
| patrickeast | lennyb_: do you mean jenkins like the infra jenkins account posting on gerrit or a jenkins server in general? | 15:54 |
| *** ChuckC_ has joined #openstack-meeting-4 | 15:54 | |
| asselin_ | #link https://github.com/Triniplex/third-party-ci-dashboard | 15:54 |
| asselin_ | patrickeast, FYI^^ | 15:55 |
| krtaylor | I think that part is fairly easy, getting the list of systems to monitor, harder | 15:55 |
| lennyb_ | patrickeast, Jenkins server in general | 15:55 |
| patrickeast | asselin_: oh nice, thanks! | 15:55 |
| asselin_ | #link http://dashboard.triniplex.com/#!/ | 15:55 |
| asselin_ | sweston had it running here, but I don't see anything now ^^ | 15:55 |
| krtaylor | actually #link https://github.com/stackforge/radar | 15:56 |
| patrickeast | lennyb_: gotcha, i was looking to catch events a bit further down the chain when comments are added to gerrit, i noticed that for my ci it would sometimes report things like NOT_REGISTERED or just ERROR from zuul and jenkins would never even know it happend | 15:56 |
| krtaylor | the requirement was to have a place where a developer (core) could go see if a system was behaving | 15:56 |
| krtaylor | this just came up with a patch that failed on a system, but was ignored and merged anyway | 15:57 |
| krtaylor | mainly because ci systems are not seen as reliable | 15:57 |
| patrickeast | yea its a problem in cinder right now | 15:57 |
| wznoinsk | patrickeast: listening to stream event using ssh may be more reliable | 15:58 |
| krtaylor | if someone could check if it was reliable, then its report could be trusted if a patch comment showed a failure | 15:58 |
| patrickeast | wznoinsk: yep, the scoreboard thing does just that with paramiko | 15:59 |
| lennyb_ | patrickeast, this issue we solved by searching ERROR/Traceback etc in /var/log/zuul and checking that Jenkins has done something in the last few hours ( statistics :) ) | 15:59 |
| *** matrohon has quit IRC | 15:59 | |
| patrickeast | lennyb_: ahh yea that would work too | 15:59 |
| krtaylor | also, radar was done without spec, not sure if this effort really requires one, but it would be good for us to unify on the work and come up with one really good solution | 15:59 |
| *** galstrom is now known as galstrom_zzz | 15:59 | |
| krtaylor | so the question is - what to do with this spec? | 16:00 |
| krtaylor | it seems to be gating this effort | 16:00 |
| patrickeast | imo there are a few big features in the spec… it should probably be broken up | 16:00 |
| krtaylor | sweston has put some fine work into it | 16:00 |
| patrickeast | some of them like the ci control channel and notifications are blocking it | 16:00 |
| patrickeast | when all we really need right now is a dash that shows us what is broken | 16:01 |
| krtaylor | I suggested that we start with a cmd line | 16:01 |
| *** galstrom_zzz is now known as galstrom | 16:01 | |
| krtaylor | oops, we are out of time | 16:01 |
| krtaylor | let's continue via email | 16:01 |
| asselin_ | agree, we should start with something simple. Keeps scope creep out of the spec. And put those in a new spec. | 16:02 |
| krtaylor | thanks everyone, great meeting! | 16:02 |
| krtaylor | #endmeeting | 16:02 |
| *** banix_ has joined #openstack-meeting-4 | 16:04 | |
| *** numan has quit IRC | 16:08 | |
| *** klamath_ has joined #openstack-meeting-4 | 16:08 | |
| *** klamath has quit IRC | 16:08 | |
| *** klamath_ has quit IRC | 16:09 | |
| *** klamath has joined #openstack-meeting-4 | 16:10 | |
| *** ja_ has quit IRC | 16:12 | |
| *** fnaval has joined #openstack-meeting-4 | 16:12 | |
| *** zz_zz_zz_ja- is now known as zz_zz_zz_zz_ja- | 16:13 | |
| *** hareeshp has quit IRC | 16:18 | |
| *** Aish has joined #openstack-meeting-4 | 16:20 | |
| *** asselin__ has joined #openstack-meeting-4 | 16:25 | |
| *** MarkAtwood has joined #openstack-meeting-4 | 16:27 | |
| *** jckasper has joined #openstack-meeting-4 | 16:27 | |
| *** asselin_ has quit IRC | 16:29 | |
| *** VW_ has quit IRC | 16:34 | |
| *** VW_ has joined #openstack-meeting-4 | 16:37 | |
| *** MarkAtwood has quit IRC | 16:40 | |
| *** jckasper has quit IRC | 16:44 | |
| *** Tcs has joined #openstack-meeting-4 | 16:45 | |
| *** jckasper has joined #openstack-meeting-4 | 16:45 | |
| *** MarkAtwood has joined #openstack-meeting-4 | 16:45 | |
| *** SridharRamaswamy has joined #openstack-meeting-4 | 16:53 | |
| *** asselin__ has quit IRC | 16:56 | |
| *** galstrom is now known as galstrom_zzz | 16:58 | |
| *** patrickeast has quit IRC | 16:59 | |
| *** s3wong has joined #openstack-meeting-4 | 17:01 | |
| yamahata | hello | 17:03 |
| SridharRamaswamy | s3wong: yamahata: hi | 17:03 |
| s3wong | hello | 17:04 |
| yamahata | #startmeeting servicevm-device-manager | 17:04 |
| openstack | Meeting started Wed Mar 4 17:04:29 2015 UTC and is due to finish in 60 minutes. The chair is yamahata. Information about MeetBot at http://wiki.debian.org/MeetBot. | 17:04 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 17:04 |
| *** openstack changes topic to " (Meeting topic: servicevm-device-manager)" | 17:04 | |
| openstack | The meeting name has been set to 'servicevm_device_manager' | 17:04 |
| yamahata | #chair SridharRamaswamy s3wong | 17:04 |
| openstack | Current chairs: SridharRamaswamy s3wong yamahata | 17:04 |
| yamahata | #topic Announcement | 17:05 |
| *** openstack changes topic to "Announcement (Meeting topic: servicevm-device-manager)" | 17:05 | |
| yamahata | any announcement? | 17:05 |
| yamahata | I suppose the notification of summit session proposal is not yet. | 17:05 |
| *** pkoniszewski has joined #openstack-meeting-4 | 17:05 | |
| s3wong | yamahata: no | 17:05 |
| SridharRamaswamy | yamahata: haven't seen any | 17:05 |
| *** banix_ has quit IRC | 17:06 | |
| yamahata | #topic Action Items from the last week | 17:06 |
| *** openstack changes topic to "Action Items from the last week (Meeting topic: servicevm-device-manager)" | 17:06 | |
| yamahata | My devstack patch is at https://github.com/yamahata/devstack/tree/snapshot-adv-svc-vm | 17:07 |
| s3wong | yamahata: nice | 17:07 |
| yamahata | It's quite old unfortunately. | 17:07 |
| yamahata | I thought I had new one, but I counldn't fine int | 17:07 |
| yamahata | I counldn't find one | 17:07 |
| yamahata | #topic Open Discussion | 17:09 |
| *** openstack changes topic to "Open Discussion (Meeting topic: servicevm-device-manager)" | 17:09 | |
| yamahata | anything to discuss? | 17:09 |
| SridharRamaswamy | folks - is there a list of functionality written down for the new scope of tacker ? | 17:10 |
| s3wong | yamahata: not from me --- sorry, day job has been busy | 17:10 |
| s3wong | SridharRamaswamy: not that I am aware of | 17:10 |
| yamahata | everyone is busy.. | 17:10 |
| SridharRamaswamy | if i may rattle out few :) | 17:11 |
| yamahata | SridharRamaswamy: I'm not aware of it. | 17:11 |
| s3wong | SridharRamaswamy: please go ahead | 17:11 |
| SridharRamaswamy | 1) catalog of service-vm images | 17:11 |
| SridharRamaswamy | 2) basic life-cycle of service-vm (start/stop) | 17:11 |
| SridharRamaswamy | 3) basic health monitoring of service-vm | 17:12 |
| SridharRamaswamy | 4) respin of service-vm on failure | 17:12 |
| SridharRamaswamy | these are the top ones | 17:12 |
| SridharRamaswamy | make sense? | 17:12 |
| s3wong | SridharRamaswamy: we need to have infra for config saving and config push (in case of resin) | 17:13 |
| s3wong | respin | 17:13 |
| SridharRamaswamy | okay .. lets add it | 17:13 |
| SridharRamaswamy | 5) maintaining VNF configuration state | 17:13 |
| s3wong | SridharRamaswamy: that is a good initial list | 17:14 |
| SridharRamaswamy | I stopped at (4) because there are remaining functions all go into being VNF specific | 17:14 |
| SridharRamaswamy | which is fine | 17:14 |
| yamahata | SridharRamaswamy: really good. | 17:14 |
| s3wong | yamahata, SridharRamaswamy: we can aim for these as initial list of functions | 17:15 |
| SridharRamaswamy | between (1) and (2) there is something missing .. atleast for me | 17:15 |
| -openstackstatus- NOTICE: Issue solved, gate slowly digesting accumulated changes | 17:15 | |
| SridharRamaswamy | the network to land the service-vm .. | 17:16 |
| *** pkoniszewski has quit IRC | 17:16 | |
| SridharRamaswamy | is that VNF specific or one separate tenent and network to host all VNFs ? | 17:16 |
| s3wong | SridharRamaswamy: the VNF spawned needs to have a network position for the tenant | 17:17 |
| s3wong | SridharRamaswamy: but for management network, should it be a common management network which provider can access them? | 17:17 |
| SridharRamaswamy | yeah, that make sense.. | 17:18 |
| s3wong | SridharRamaswamy, yamahata: from yamahata's point last time, we will use provider-net from Nova to communicate with all the VNFs | 17:19 |
| SridharRamaswamy | that is for mgmt network access for the VNFs, correct? | 17:20 |
| s3wong | this also will allow admin/provider to access/manage VNFs created by tenants | 17:20 |
| yamahata | Yes. For now provider-net is practical and easy way. | 17:20 |
| SridharRamaswamy | how about the tenant workload traffic going to go to a VNF ? | 17:21 |
| s3wong | SridharRamaswamy: yes, we will use it for admin access, as well as config fetch / push | 17:21 |
| s3wong | SridharRamaswamy: that needs to be hot-plug | 17:21 |
| s3wong | (which I hope is working well :-) ) | 17:21 |
| SridharRamaswamy | it is working for us :) | 17:21 |
| yamahata | hot-plug is issue. | 17:21 |
| yamahata | SridharRamaswamy: cool. | 17:22 |
| yamahata | Probably using l2-gateway or VLAN-aware-VM stuff as workaround. | 17:22 |
| *** pmesserli has quit IRC | 17:22 | |
| SridharRamaswamy | the question i'm leading to is .. what kind of metadata we need to capture in the VNF catalog per vnf | 17:22 |
| yamahata | I proposed blueprint to allow to change tenant owner of port, it was rejected. | 17:23 |
| *** pmesserli has joined #openstack-meeting-4 | 17:23 | |
| s3wong | yamahata: you mean any additional networks that tenant wants the VNF to connect to, we will just hook it up via l2-gateway? | 17:23 |
| yamahata | s3wong: correct. and hot-plug happens at l2-gateway so that servicevm doesn't need hot-plug | 17:23 |
| *** matrohon has joined #openstack-meeting-4 | 17:24 | |
| s3wong | SridharRamaswamy: outside of the service type, associated config, network position, image...etc? | 17:24 |
| SridharRamaswamy | s3wong: okay | 17:24 |
| s3wong | SridharRamaswamy: we will likely add more (a lot more) as we evolve | 17:25 |
| SridharRamaswamy | s3wong: sure, of course | 17:25 |
| SridharRamaswamy | yamahata: thats an interesting proposal .. | 17:25 |
| yamahata | the spec, https://review.openstack.org/#/c/135520/ was too radical. it needs re-think. | 17:27 |
| s3wong | SridharRamaswamy, yamahata: that would work if the only insertion we would do is putting the VNFs in Neutron networks | 17:27 |
| *** pmesserli has quit IRC | 17:27 | |
| s3wong | -2 by salv-orlando | 17:28 |
| s3wong | yamahata: assuming hot plug is working, wouldn't a way to allow VMs to start without connecting to a Neutron network useful? | 17:30 |
| s3wong | yamahata: as well as allowing Nova instance to 'unplug' from Neutron network | 17:30 |
| yamahata | s3wong: yes, it's quite useful. | 17:30 |
| s3wong | yamahata: and perhaps not as controversial :-) | 17:31 |
| yamahata | anything else to discuss? | 17:34 |
| s3wong | yamahata: that's it for me | 17:34 |
| *** ajmiller_ has joined #openstack-meeting-4 | 17:34 | |
| SridharRamaswamy | thats it for me.. | 17:34 |
| s3wong | SridharRamaswamy: although you may want to put the list of initial functionality on wiki page also | 17:34 |
| SridharRamaswamy | will do | 17:34 |
| yamahata | SridharRamaswamy: cool. | 17:34 |
| s3wong | SridharRamaswamy: thanks | 17:34 |
| yamahata | thanks everyone. bye | 17:35 |
| SridharRamaswamy | bye folks | 17:35 |
| yamahata | #endmeeting | 17:35 |
| *** openstack changes topic to "Zuul check queue stuck due to reboot maintenance window at one of our cloud providers - no need to recheck changes at the moment, they won't move forward." | 17:35 | |
| openstack | Meeting ended Wed Mar 4 17:35:07 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:35 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/servicevm_device_manager/2015/servicevm_device_manager.2015-03-04-17.04.html | 17:35 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/servicevm_device_manager/2015/servicevm_device_manager.2015-03-04-17.04.txt | 17:35 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/servicevm_device_manager/2015/servicevm_device_manager.2015-03-04-17.04.log.html | 17:35 |
| *** eghobo has joined #openstack-meeting-4 | 17:35 | |
| s3wong | bye | 17:35 |
| *** s3wong has quit IRC | 17:35 | |
| *** Aish has quit IRC | 17:35 | |
| *** ajmiller has quit IRC | 17:36 | |
| *** Tcs has left #openstack-meeting-4 | 17:38 | |
| *** galstrom_zzz is now known as galstrom | 17:39 | |
| *** SumitNaiksatam has joined #openstack-meeting-4 | 17:44 | |
| *** pmesserli has joined #openstack-meeting-4 | 17:44 | |
| *** Aish has joined #openstack-meeting-4 | 18:05 | |
| *** ajmiller has joined #openstack-meeting-4 | 18:06 | |
| *** ajmiller_ has quit IRC | 18:08 | |
| *** SumitNaiksatam has quit IRC | 18:10 | |
| *** igordcard has quit IRC | 18:11 | |
| *** vishwanathj has joined #openstack-meeting-4 | 18:11 | |
| *** kobis has quit IRC | 18:13 | |
| *** banix has joined #openstack-meeting-4 | 18:31 | |
| *** SumitNaiksatam has joined #openstack-meeting-4 | 18:35 | |
| *** ivar-lazzaro has joined #openstack-meeting-4 | 18:38 | |
| *** ivar-lazzaro has quit IRC | 18:40 | |
| *** ivar-lazzaro has joined #openstack-meeting-4 | 18:41 | |
| *** VW_ has quit IRC | 18:42 | |
| *** ChuckC_ has quit IRC | 18:46 | |
| *** VW_ has joined #openstack-meeting-4 | 18:59 | |
| *** SridharRamaswamy has quit IRC | 19:00 | |
| *** eghobo has quit IRC | 19:01 | |
| *** klamath has quit IRC | 19:02 | |
| *** eghobo has joined #openstack-meeting-4 | 19:05 | |
| *** dannywilson has joined #openstack-meeting-4 | 19:07 | |
| *** dannywilson has quit IRC | 19:09 | |
| *** dannywilson has joined #openstack-meeting-4 | 19:10 | |
| *** dannywilson has quit IRC | 19:10 | |
| *** dannywilson has joined #openstack-meeting-4 | 19:11 | |
| *** claired has joined #openstack-meeting-4 | 19:19 | |
| *** eghobo has quit IRC | 19:22 | |
| *** nkrinner has joined #openstack-meeting-4 | 19:24 | |
| *** s3wong has joined #openstack-meeting-4 | 19:25 | |
| *** SridharRamaswamy has joined #openstack-meeting-4 | 19:26 | |
| *** eghobo has joined #openstack-meeting-4 | 19:30 | |
| *** s3wong has quit IRC | 19:37 | |
| *** s3wong has joined #openstack-meeting-4 | 19:37 | |
| *** ppetit has quit IRC | 19:38 | |
| *** pkoniszewski has joined #openstack-meeting-4 | 19:41 | |
| *** wojdev has joined #openstack-meeting-4 | 19:41 | |
| *** hareeshp has joined #openstack-meeting-4 | 19:43 | |
| *** matrohon has quit IRC | 19:46 | |
| *** Rockyg has joined #openstack-meeting-4 | 19:51 | |
| *** hareeshp has quit IRC | 19:55 | |
| *** masteinhauser has joined #openstack-meeting-4 | 19:55 | |
| *** banix has quit IRC | 19:56 | |
| *** wojdev has quit IRC | 19:57 | |
| *** jokke_ has joined #openstack-meeting-4 | 19:58 | |
| *** barrett has joined #openstack-meeting-4 | 19:59 | |
| *** eghobo has quit IRC | 20:00 | |
| *** bknudson has joined #openstack-meeting-4 | 20:00 | |
| Rockyg | Hi. Who's here for the log working group meeting? | 20:01 |
| jokke_ | o/ | 20:01 |
| *** ppetit_ has joined #openstack-meeting-4 | 20:01 | |
| bknudson | hi | 20:01 |
| nkrinner | heya. me too | 20:01 |
| jokke_ | yeii ... it's still not just two of us :D | 20:01 |
| masteinhauser | o/ | 20:02 |
| Rockyg | Yay! Ok. gonna start... | 20:02 |
| barrett | I am | 20:02 |
| Rockyg | #startmeeting log-wg | 20:02 |
| openstack | Meeting started Wed Mar 4 20:02:17 2015 UTC and is due to finish in 60 minutes. The chair is Rockyg. Information about MeetBot at http://wiki.debian.org/MeetBot. | 20:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 20:02 |
| *** openstack changes topic to " (Meeting topic: log-wg)" | 20:02 | |
| openstack | The meeting name has been set to 'log_wg' | 20:02 |
| *** wojdev has joined #openstack-meeting-4 | 20:02 | |
| *** eghobo has joined #openstack-meeting-4 | 20:02 | |
| dhellmann | o/ | 20:03 |
| Rockyg | #topic intros | 20:03 |
| *** openstack changes topic to "intros (Meeting topic: log-wg)" | 20:03 | |
| Rockyg | I'd like to get everyone on similar pages, so if you can do a quick intro of who you are, what part of the community and what your top logging issue/goal is, that will get us rolling | 20:04 |
| Rockyg | No novels, please, though | 20:04 |
| dhellmann | Hi, I'm Doug Hellmann, Oslo PTL and TC member. | 20:05 |
| bknudson | Brant Knudson -- Keystone core reviewer -- hoping to have logs that can be used to debug issues. | 20:05 |
| masteinhauser | Hi, I'm Myles Steinhauser, Engineer at Blue Box Group. Automated alerting and Ops of primary interest. | 20:05 |
| nkrinner | hi, my name is Nanuk Krinner, i am a software developer for suse and work on openstack. i attended the kickoff meeting at the kilo summit and want to help improving the logging situation | 20:05 |
| barrett | Carol Barrett -- Win The Enterprise WG -- monitoring/logging is a top issue for Enterprise Deployment of OpenStack | 20:05 |
| Rockyg | Rocky Grober -- product group, evangalist, QA -- consistent logs so problems can be quickly identified | 20:06 |
| jokke_ | Erno Kuvaja - Glance Core reviewer - and my goal is to have logs that makes it possible for us to debug and support/ops to support openstack in real life environments | 20:06 |
| Rockyg | kewl. let's discuss state of of logging and community wrt logging | 20:07 |
| ppetit_ | Patrick Petit, working on OSt logs anaytics project in Fuel. Goal is to bring logs at a level of consistency can be effectively used for operations | 20:08 |
| Rockyg | #topic cureent state | 20:08 |
| *** openstack changes topic to "cureent state (Meeting topic: log-wg)" | 20:08 | |
| *** swann has joined #openstack-meeting-4 | 20:08 | |
| *** jecarey_ has joined #openstack-meeting-4 | 20:08 | |
| bknudson | the keystone logs are useless. | 20:08 |
| jokke_ | "~agreed not great" :P | 20:09 |
| nkrinner | yesterday | 20:09 |
| Rockyg | dhellman and Oslo have created a log library that can help with getting consistent formats. Especially headers, etc | 20:09 |
| nkrinner | sorry | 20:09 |
| jokke_ | I'd like to hear if someone thinks that there is actuallyt project out there that has great logging | 20:09 |
| bknudson | keystone uses the log library... doesn't help us log what's needed. | 20:10 |
| Rockyg | I just looked at keystone's log docs and they seem to have vanished. Last cycle, they looked to be promoting oslo_log | 20:10 |
| dhellmann | it might be more constructive to talk about some specific deficiencies | 20:10 |
| Rockyg | dhellman: good point | 20:10 |
| dhellmann | I assume everyone had a chance to comment on http://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html for example? | 20:11 |
| jokke_ | tbh I don't think our biggest problem is even near oslo.log | 20:11 |
| *** yamahata has quit IRC | 20:11 | |
| nkrinner | yesterday i worked on a heat issue where the logs provided no information about why starting of a heat stack failed. the other day when i had issues with nova they were very helpful though. the state of logs varies over the projects. | 20:11 |
| bknudson | here's a specific deficiency, at least on keystone I think it's needed to have some way to correlate a request with the error response, but we don't have that, either in the client or the logs or the server. | 20:11 |
| *** pkoniszewski has quit IRC | 20:12 | |
| jokke_ | bknudson: I'd say :%s/or/and/ | 20:12 |
| bknudson | y, I also have specific things where I would have like the logs to show me what I was doing wrong and there was nothing, had to resort to the debugger. | 20:13 |
| dhellmann | nkrinner: I hope you filed a bug? | 20:13 |
| jokke_ | bknudson: I think one of the most important things is to be able to correlate the user action to possible middle man logs and at the end to the actual service logs | 20:13 |
| dhellmann | bknudson: that's a good one, and I think there's another spec related to that... | 20:13 |
| nkrinner | dhellmann: not yet, will do so | 20:13 |
| Rockyg | I believe its a glance spec | 20:13 |
| masteinhauser | log correlation via req-* and tracking through the entire request pipeline and fanout would be incredibly useful. | 20:13 |
| dhellmann | #link https://review.openstack.org/156508 | 20:13 |
| dhellmann | nkrinner: please do! | 20:14 |
| bknudson | jokke_: actually, in keystone it's probably easier since there aren't middlemen... the middlemen is more difficult. | 20:14 |
| ppetit_ | w.r.t correlating on request id a first step would be to have all projects sync to oslo.log | 20:14 |
| bknudson | for some reason https://review.openstack.org/#/c/156508/ says cinder... not sure why it would be specific. | 20:14 |
| jokke_ | bknudson: nope ... every other service has that middleman towards keystone ;) | 20:14 |
| Rockyg | nkrinner: were debug logs of any help or you had to do it interactively? | 20:14 |
| dhellmann | bknudson: I think they just wanted to start there | 20:15 |
| Rockyg | sorry, bknudson... | 20:15 |
| dhellmann | ppetit_: all of the projects are using *a* version of the oslo logging code, except possibly swift. The work we did this cycle was to turn that into a library, but it has been in the incubator for a while. | 20:16 |
| bknudson | Rockyg: actually, debug was enabled, and the debug log didn't help... I know where to add the debug log now. | 20:16 |
| jokke_ | bknudson: iirc that cinder spec was made for one project just to have some scope for it (same as I wrote the start of error code spec under glance just because it was familiar environment to get started with) | 20:16 |
| nkrinner | Rockyg: i looked at the debug logs and did not find anything helpful there. I don't have them here, but will look at them and file a bug with relevant information | 20:16 |
| Rockyg | dhellmann: would it be reasonable to take the cinder spec and create a cross spec for all projects? | 20:16 |
| dhellmann | Rockyg: yes, that was the feedback ttx left after the cross-project meeting discussed the spec | 20:17 |
| bknudson | I would love to see https://review.openstack.org/#/c/156508/ for keystone... that's one part of it... other parts are 1) logging the request ID in keystone log, 2) displaying request ID in CLI | 20:17 |
| masteinhauser | dhellmann: glad you mentioned swift, that has been incredibly painful for us dealing with logging. | 20:17 |
| dhellmann | masteinhauser: I'm not sure of the current situation there, I haven't looked in a while. | 20:18 |
| dhellmann | masteinhauser: have you provided that feedback to the swift team? | 20:18 |
| masteinhauser | dhellmann: I don't know the exact refs we are running in production, I can check. | 20:18 |
| ppetit_ | dhellmann the problem with project using the incubator version is that the request id and tenant id are not properly rendered | 20:18 |
| masteinhauser | dhellmann: I have not, yet. Finally diving more into the community aspects. | 20:18 |
| *** wojdev has quit IRC | 20:18 | |
| Rockyg | can we get a volunteer to generate the crossproject version of spec for https://review.openstack.org/#/c/156508/ ? | 20:19 |
| dhellmann | ppetit_: ok, that problem may have been fixed but not synced, but at this point we do want projects to start using the library. Full adoption may not happen until L though. | 20:19 |
| dhellmann | Rockyg: someone should work with the original author on that, rather than starting a new spec | 20:19 |
| ppetit_ | It should happen before L IMO | 20:19 |
| ppetit_ | Jay Pipes has an action item to make that happen | 20:20 |
| dhellmann | ppetit_: I would have liked for that to happen, too, but there were some delays early in the cycle and we're only a few weeks away from feature freeze at this point | 20:20 |
| Rockyg | dhellman: agreed. | 20:20 |
| dhellmann | ppetit_: I'm not saying it *shouldn't* just that we shouldn't count on it. :-) | 20:20 |
| Rockyg | but a new version of the lib was just released. it should help? | 20:20 |
| dhellmann | full adoption of the library is one of my goals for L, though | 20:20 |
| jokke_ | I'd be happy if the bikeshedding around the X-spec is done by start of Liberty so people get to work on it right away | 20:21 |
| Rockyg | dhellman: noticed that the oslo log docs are getting fleshed out, so should make it easier for developers to implement against | 20:22 |
| dhellmann | jokke_: the best way to make that happen is to participate in the conversation and push it to conclusion | 20:22 |
| Rockyg | Abhishek Kekane is the owner of 156508 | 20:22 |
| dhellmann | Rockyg: thanks, I think they're looking fairly good now | 20:22 |
| Rockyg | agreed. | 20:22 |
| Rockyg | I would love to see some code snippets, though. So devs can cut and paste | 20:23 |
| ppetit_ | beyond consistency of format we are certainly facinf also a lack of consistency and effectivness of the logs produced at INFO level for the consumption of operators… | 20:24 |
| Rockyg | Also want to point out that I saw in passing a question on QA yesterday as to whether oslo_log should be used for tests now. We should socialize that idea. | 20:24 |
| *** hareeshp has joined #openstack-meeting-4 | 20:24 | |
| dhellmann | ppetit_: agreed, that is a big goal for sdague's spec linked earlier | 20:24 |
| *** wojdev has joined #openstack-meeting-4 | 20:24 | |
| Rockyg | Again, I think examples of what should be in INFO go a long ways to getting devs to do it. | 20:24 |
| dhellmann | Rockyg: there are fixtures to hook up logs for tests, if that's what you mean? | 20:24 |
| dhellmann | I think that's covered by http://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html isn't it? | 20:25 |
| Rockyg | I think so. It was a one liner | 20:25 |
| masteinhauser | ppetit_: agreed, we are running all production stacks at info and some at debug logging consistently to capture usable error output. I can provide examples with some research. | 20:26 |
| jokke_ | ppetit_ & dhellmann: I think that sdague's spec is good start to that direction, then we just need to beat that scaryness out of WARN ;) | 20:26 |
| Rockyg | log-guidelines have general examples. I think we need to get a member of each projecyt to own log improvement | 20:26 |
| dhellmann | jokke_: right, the next step there may be to open bugs against projects with specific cases of where logging is not matching those guidelines | 20:27 |
| dhellmann | Rockyg: having an owner per project makes sense | 20:27 |
| ppetit_ | We can taake a share of that effort | 20:27 |
| dhellmann | ppetit_: ++ | 20:28 |
| jokke_ | I think the bigger problem is to get people out of their old habbits and "It's fine on devstack" mentality | 20:28 |
| Rockyg | #action get log message owner for each project and add to Theirry's page tracking them | 20:28 |
| dhellmann | jokke_: this is going to take a fair amount of education, which will take time, but we can improve the current situation as we go, too | 20:28 |
| dhellmann | Rockyg: "Thierry's page"? | 20:29 |
| jokke_ | dhellmann: I totally agree | 20:29 |
| bknudson | #link https://wiki.openstack.org/wiki/CrossProjectLiaisons | 20:29 |
| bknudson | ? | 20:29 |
| dhellmann | bknudson: ah, yeah, that makes sense | 20:29 |
| dhellmann | Rockyg: does this group have a home page in the wiki? | 20:29 |
| jokke_ | dhellmann: I'd really like to hear about the ideas how to do that ... one person onboard does not make miracles if most active devs and cores disagrees :( | 20:30 |
| Rockyg | https://wiki.openstack.org/wiki/LogWorkingGroup | 20:30 |
| Rockyg | thanks bknudson. You beat me to it | 20:30 |
| jokke_ | like just making some minor adjustments on Glance was half a year long rocky road to fight through | 20:30 |
| *** dims has joined #openstack-meeting-4 | 20:30 | |
| bknudson | Are there cores or projects that don't think logging is useful? | 20:30 |
| bknudson | obviously it will slow development to -1 over logging issues. | 20:31 |
| Rockyg | One person does not, but getting the common devref and fixing project devref gets newbies started | 20:31 |
| dhellmann | #link https://wiki.openstack.org/wiki/CrossProjectLiaisons#Logging_Working_Group | 20:31 |
| jokke_ | bknudson: it's not about logging being useful, it's more about being too verbose is ok or hiding stuff under DEBUG is ok as everyone runs their clouds on debug, right | 20:32 |
| Rockyg | I noticed some projects have hacking rules for logs. Propose development of common hacking rules? | 20:32 |
| dhellmann | jokke_: around here we have to lead by example | 20:32 |
| jokke_ | dhellmann: ++ | 20:32 |
| ppetit_ | BTW we are releasing a tool Heka / ElasticSearch / Kibana with pre-configured parsers and templates that that help debugging logs. Its being packaged as Fuel plugin but can be easily extracted | 20:32 |
| dhellmann | Rockyg: most of those rules have to do with enforcing the translation markers, don't they? I'm not sure we can use code analysis to enforce the other guidelines. | 20:32 |
| Rockyg | Translation is the big one | 20:33 |
| jokke_ | and important | 20:33 |
| dhellmann | I do really believe that if we identify deficiencies and actually start fixing them then people will be on board when they see the improvements. | 20:33 |
| dhellmann | So let's focus on that before making more rules or tools. | 20:33 |
| Rockyg | dhellmann: is the global requirement to set log style to syslog in? | 20:33 |
| dhellmann | I'm not aware of that as a requirement? | 20:33 |
| Rockyg | requirements.txt Sorry. I know you had put in a patch | 20:34 |
| Rockyg | So the format could be set in global instead of project by project | 20:34 |
| dhellmann | oslo.log is in the global requirements list, yes. Not all projects are currently using the library version of the oslo logging module, though. | 20:34 |
| jokke_ | Rockyg: requirements is not enforcing configs, but providing list of required dependencies | 20:35 |
| Rockyg | #action Identify guidelines that can make sense to add to Hacking | 20:35 |
| dhellmann | we can work on oslo.log adoption and cleaning up the info vs. debug level issues in parallel, but we will need someone to produce those patches | 20:35 |
| Rockyg | Dhellman: right. Got a list of those here: https://etherpad.openstack.org/p/Log-Rationalization | 20:36 |
| bknudson | it's been on my list o' things to do in keystone for a long time. | 20:36 |
| dhellmann | #link https://etherpad.openstack.org/p/Log-Rationalization | 20:36 |
| bknudson | but it's a lot of work and also not the most interesting. | 20:37 |
| jokke_ | Perhaps we should move on, I think we had other topics in the list still | 20:37 |
| Rockyg | under Developer Docs but needs updating... | 20:37 |
| dims | bknudson: nova is now oslo.log enabled, we hit a few bumps, one bump still in progress (patch is in nova) | 20:37 |
| dhellmann | jokke_: ++ | 20:37 |
| *** banix has joined #openstack-meeting-4 | 20:38 | |
| bknudson | dims: I was talking about switching to oslo.log ... that's easy ... was talking about info vs debug issues & useful logging in general. | 20:38 |
| Rockyg | #topic Error codes | 20:38 |
| *** openstack changes topic to "Error codes (Meeting topic: log-wg)" | 20:38 | |
| * dhellmann looks for his copy of the agenda | 20:38 | |
| bknudson | *wasn't* | 20:38 |
| dims | bknudson: ack | 20:38 |
| Rockyg | I started a x-project version of jokke's sxpec | 20:38 |
| jokke_ | this seems to be topic that divides people more than request IDs | 20:39 |
| Rockyg | #link https://review.openstack.org/#/c/127482 | 20:39 |
| bknudson | where's the x-project version? | 20:40 |
| Rockyg | I think a big issue is to separate the layers of the system so folks understand better what the message focus is bknudson: still working on it. Focus to get it to review befor EOW | 20:40 |
| *** eghobo has quit IRC | 20:41 | |
| bknudson | Rockyg: ok, thanks. | 20:41 |
| jokke_ | Rockyg: let me know if you are/get stuck with it | 20:42 |
| *** wojdev has quit IRC | 20:42 | |
| Rockyg | I figure the right way to do these sorts of xproject specs is to do them, then link to the project versions, which will have the right level of detail for devs | 20:42 |
| bknudson | I thought this was going to define an error document or header? | 20:42 |
| *** bnemec has joined #openstack-meeting-4 | 20:42 | |
| jokke_ | bknudson: I still do not know where that header idea has came from nor really a use case for it | 20:43 |
| Rockyg | So, the thinking is: for every error, create a code with a summary description. Then the payload will have specifics of instance, etc | 20:43 |
| bknudson | I'm not a fan of numeric codes since it's hard to remember what the mapping is. | 20:43 |
| Rockyg | #link http://www.faqs.org/rfcs/rfc5424.html is syslog spec | 20:44 |
| dhellmann | Rockyg: +1 on linking from project blueprints to the cross-project specs | 20:44 |
| jokke_ | bknudson: they tend to be easier to remember than uuids and takes less space than the camel text ... the space is limited after all | 20:44 |
| Rockyg | The message format includes "MSGID" | 20:45 |
| bknudson | store it in the cloud. | 20:45 |
| Rockyg | We are going to split the code into Proj,component, then number | 20:45 |
| Rockyg | So, three letters for the project is the first part of the code | 20:46 |
| jokke_ | bknudson: the whole point for those is exactly that ... enabling the possibility of building knowledge bases around our erroring | 20:46 |
| *** wojdev has joined #openstack-meeting-4 | 20:46 | |
| bknudson | 3 letters should be enough for anyone. | 20:46 |
| Rockyg | Yup. | 20:46 |
| Rockyg | I'll get the spec out pdq | 20:46 |
| Rockyg | Then it will make more sense. | 20:46 |
| dhellmann | yeah, let's save the format discussion for the spec review | 20:47 |
| jokke_ | ++ | 20:47 |
| jokke_ | should we move on to Ops meetup | 20:47 |
| Rockyg | also, with the way syslog works, one log message could be encapsulated in another if we wanted to cascade to track the effects | 20:48 |
| Rockyg | #topic Ops meetup | 20:48 |
| *** openstack changes topic to "Ops meetup (Meeting topic: log-wg)" | 20:48 | |
| Rockyg | I can be there, but I've be waffling. Should I? | 20:48 |
| jokke_ | so we have some real ideas, specificly around req ids and error codes ... I'd like to get those two speks even if not ready out there and the links to those reviews to the ops for feedback | 20:49 |
| Rockyg | I think if we have referrer-id and error code specs in review, we can get ops input | 20:49 |
| dhellmann | is there already an agenda for that meetup, or is it being put together on site? | 20:49 |
| Rockyg | I also think we need to socialize the logging guidelines spec to ops more. | 20:49 |
| jokke_ | ++ | 20:50 |
| Rockyg | I think they could create a spec or two that will make the guidelines more useful for them | 20:50 |
| Rockyg | dhellmann: will you be at the meetup? | 20:51 |
| dhellmann | Rockyg: no, I'm afraid not | 20:51 |
| Rockyg | Any other Oslo core? | 20:51 |
| dhellmann | I haven't had anyone say they're going | 20:51 |
| Rockyg | I'd love to find a developer who can step up and become the lead for oslo-log, but that's not a small order | 20:52 |
| * jokke_ volunteers dhellmann :P | 20:52 | |
| dhellmann | I think dims and I have that covered. I'm more concerned about the work on the other projects. | 20:52 |
| Rockyg | Yeah. Like he doesn't have enough hats to wear... | 20:52 |
| jokke_ | dhellmann: ++ | 20:52 |
| dhellmann | we don't have that much work to do on the library itself, afaik | 20:52 |
| Rockyg | dhellmann: I think you're right. It's really close at this point | 20:53 |
| jokke_ | dhellmann: agreed and as said in Paris, if there comes something we need to get done, I'm more than happy trying to help | 20:53 |
| dhellmann | jokke_: thanks, I'll keep that in mind when the time comes | 20:54 |
| Rockyg | OK. So, maybe I can go to the meetup and try to recruit devs who turn up to champion oslo-log on their projects? | 20:55 |
| Rockyg | If they are there for Ops, they care. | 20:55 |
| *** SridharRamaswamy has quit IRC | 20:55 | |
| dims | Rockyg: +1 :) | 20:55 |
| dhellmann | Rockyg: that's a good idea. I would also bring it up as a need during the cross-project weekly meeting, and see about getting PTLs to help identify liaisons | 20:55 |
| Rockyg | Carol, will you be there? | 20:56 |
| jokke_ | Rockyg: that and any input from the ops side is valued ... if you get the message through that we want to do this right, that would be great | 20:56 |
| Rockyg | Folks focused on WTE will want good logs | 20:56 |
| Rockyg | OK. I'm there. | 20:56 |
| Rockyg | Meet with boss this afternoon and I'll let him know I'm going... | 20:56 |
| jokke_ | :D | 20:56 |
| *** wojdev has quit IRC | 20:57 | |
| Rockyg | #topic priorities | 20:57 |
| *** openstack changes topic to "priorities (Meeting topic: log-wg)" | 20:57 | |
| dhellmann | fwiw, I don't think the folks working on these logging patches need to necessarily be cores on the projects, so we should be able to recruit from a wider pool than might be at the midcycles | 20:57 |
| jokke_ | dhellmann: ++ | 20:57 |
| Rockyg | I think refactoring log messages will be good low hanging fruit | 20:58 |
| bknudson | lots of the problems with keystone logging are actually problems with the whole design of the keystone code. | 20:58 |
| jokke_ | I think the priority nro 1. should be educating about the guideline spec nro 2. getting feedback and hammer down those two other specs by start of Liberty | 20:58 |
| Rockyg | 3. get liasons from projects | 20:58 |
| jokke_ | bknudson: that's big problem on other projects as well. It's not easy to figure out what should be logged and where | 20:59 |
| dhellmann | bknudson: that would make it harder for a new contributor to make improvments :-/ | 20:59 |
| Rockyg | Maybe we should identify the log messages that are good. It might be a shorter list :P | 21:00 |
| jokke_ | Well I learned hell of a lot while refactoring the Glance logs :) Have to understand what is going on to be able to do meaningful logging | 21:00 |
| bknudson | jokke_: exactly. | 21:00 |
| Rockyg | I think that might be why the one dev wanted to start with APIs. Since they're restful, there's less to tweak. Theoretically. | 21:01 |
| bknudson | agree with the priorities mentioned here. | 21:01 |
| Rockyg | Not actually. | 21:01 |
| jokke_ | do we have any other burning priorities? Are we agreeing on these? (We're running out of time) | 21:02 |
| dhellmann | we're actually over time by a couple of minutes | 21:02 |
| jokke_ | is there Q behind the door already? | 21:02 |
| jokke_ | :) | 21:02 |
| dhellmann | jokke_: maybe you can start a ML thread about priorities? | 21:02 |
| bknudson | I didn't even know there was a meeting-4 | 21:02 |
| jokke_ | dhellmann: will do | 21:02 |
| *** VW_ has quit IRC | 21:03 | |
| *** VW__ has joined #openstack-meeting-4 | 21:03 | |
| Rockyg | #action Priorities: 1. Education around Logging Guidelines 2. Spec out and feedback for error code spec and referrer id 3. Get project liaisons for Log Working Group | 21:03 |
| *** VW__ has quit IRC | 21:03 | |
| jokke_ | #action jokke will bring up ML thread around priorities agreed | 21:03 |
| Rockyg | Anything else? | 21:03 |
| Rockyg | Good meeting, guys! | 21:03 |
| *** VW_ has joined #openstack-meeting-4 | 21:03 | |
| jokke_ | Thanks all! | 21:03 |
| nkrinner | thanks everybody | 21:04 |
| dhellmann | thanks! | 21:04 |
| Rockyg | #endmeeting | 21:04 |
| *** openstack changes topic to "Zuul check queue stuck due to reboot maintenance window at one of our cloud providers - no need to recheck changes at the moment, they won't move forward." | 21:04 | |
| openstack | Meeting ended Wed Mar 4 21:04:15 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:04 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/log_wg/2015/log_wg.2015-03-04-20.02.html | 21:04 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/log_wg/2015/log_wg.2015-03-04-20.02.txt | 21:04 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/log_wg/2015/log_wg.2015-03-04-20.02.log.html | 21:04 |
| *** jokke_ has left #openstack-meeting-4 | 21:04 | |
| *** nkrinner has quit IRC | 21:05 | |
| *** swann has left #openstack-meeting-4 | 21:09 | |
| *** banix has quit IRC | 21:09 | |
| *** MarkAtwood has quit IRC | 21:16 | |
| *** SridharRamaswamy has joined #openstack-meeting-4 | 21:18 | |
| *** wojdev has joined #openstack-meeting-4 | 21:20 | |
| *** rfolco has quit IRC | 21:20 | |
| *** hareeshp has quit IRC | 21:21 | |
| *** Rockyg has quit IRC | 21:26 | |
| *** wojdev has quit IRC | 21:30 | |
| *** ChuckC has joined #openstack-meeting-4 | 21:32 | |
| *** s3wong has quit IRC | 21:34 | |
| *** s3wong has joined #openstack-meeting-4 | 21:40 | |
| *** banix has joined #openstack-meeting-4 | 21:45 | |
| *** wojdev has joined #openstack-meeting-4 | 21:45 | |
| *** dhellmann has quit IRC | 21:52 | |
| krtaylor | #startmeeting third-party | 21:53 |
| openstack | Meeting started Wed Mar 4 21:53:57 2015 UTC and is due to finish in 60 minutes. The chair is krtaylor. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:53 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:53 |
| *** openstack changes topic to " (Meeting topic: third-party)" | 21:53 | |
| openstack | The meeting name has been set to 'third_party' | 21:54 |
| krtaylor | NOTE: this is a replay from the meeting held earlier today at 1500UTC, in order for it to be logged | 21:54 |
| krtaylor | <asselin_> hi | 21:54 |
| krtaylor | <ja_> morning | 21:54 |
| krtaylor | <krtaylor> hm, no meeting services? | 21:54 |
| krtaylor | <wznoinsk> hi | 21:54 |
| krtaylor | <mmedvede> o/ | 21:54 |
| krtaylor | * patrickeast (~patrick.e@50-205-1-130-static.hfc.comcastbusiness.net) has joined #openstack-meeting-4 | 21:54 |
| krtaylor | <krtaylor> we aren't logging for some reason | 21:55 |
| krtaylor | <krtaylor> hm, well I guess I cut/paste the log after the meeting to somewhere | 21:55 |
| krtaylor | <krtaylor> who's here for third-party-ci-wg meeting? :) | 21:55 |
| krtaylor | <patrickeast> o/ | 21:55 |
| krtaylor | <ja_> here for 3p | 21:55 |
| krtaylor | <asselin_> hi | 21:55 |
| krtaylor | <krtaylor> nope, that didnt work either | 21:55 |
| krtaylor | <krtaylor> oh well | 21:55 |
| krtaylor | <wznoinsk> o/ | 21:55 |
| krtaylor | <krtaylor> hey everybody | 21:55 |
| krtaylor | <krtaylor> I guess we'll carry on and see about restarting the service later | 21:55 |
| krtaylor | <krtaylor> so, in case it is working, but being silent | 21:55 |
| krtaylor | <krtaylor> here is the link for the agenda | 21:55 |
| krtaylor | <krtaylor> #link https://wiki.openstack.org/wiki/Meetings/ThirdParty#3.2F4.2F15_1500_UTC | 21:55 |
| krtaylor | <krtaylor> pretty light agenda today, but we can talk about some of the work being done | 21:55 |
| krtaylor | <krtaylor> no system being highlighted today | 21:55 |
| krtaylor | <krtaylor> let me know if you'd like to share what you are testing and how, problems you fixed, tools you built, etc | 21:55 |
| krtaylor | <krtaylor> I'll get your system scheduled for a future meeting | 21:55 |
| krtaylor | * hareeshp (hareeshp@nat/cisco/x-qsvcmndsyksqnxtv) has joined #openstack-meeting-4 | 21:55 |
| krtaylor | <krtaylor> any volunteers? | 21:55 |
| krtaylor | <wznoinsk> I'd like to present our Intel Networking CI | 21:55 |
| krtaylor | <wznoinsk> next meeting (in 2weeks) works for me | 21:55 |
| krtaylor | <krtaylor> wznoinsk, excellent | 21:55 |
| krtaylor | <krtaylor> I'll put you on the agenda | 21:55 |
| krtaylor | * yamahata (~yamahata@c-67-160-193-250.hsd1.ca.comcast.net) has joined #openstack-meeting-4 | 21:55 |
| krtaylor | <krtaylor> wznoinsk, have you seen some of the previous highlight discussions? | 21:55 |
| krtaylor | * yamahata has quit (Read error: Connection reset by peer) | 21:55 |
| krtaylor | * yamahata (~yamahata@c-67-160-193-250.hsd1.ca.comcast.net) has joined #openstack-meeting-4 | 21:55 |
| krtaylor | <wznoinsk> yes, two and one from patrickeast last time | 21:56 |
| krtaylor | <krtaylor> wznoinsk, great, I'll contact you and answer any questions about format | 21:56 |
| krtaylor | <wznoinsk> if there's any formal way of presenting I think we can talk about it after the main topics | 21:56 |
| krtaylor | <wznoinsk> good, thanks | 21:56 |
| krtaylor | <krtaylor> I need to create a template for ideas to share, nothing too format | 21:56 |
| krtaylor | <krtaylor> formal | 21:56 |
| krtaylor | * asselin_ will step away for a few minutes | 21:56 |
| krtaylor | <krtaylor> but some ideas on what would be useful to share with others | 21:56 |
| krtaylor | <wznoinsk> I'll review the previous presentation as well again | 21:56 |
| krtaylor | <krtaylor> #action krtaylor to put together a template for highlighting TPWG systems, suggestions for information to share | 21:56 |
| krtaylor | <wznoinsk> but basically I'll be focusing on dockerizing the CI as we did | 21:56 |
| krtaylor | * pkoniszewski has quit (Ping timeout: 244 seconds) | 21:56 |
| krtaylor | <krtaylor> wznoinsk, excellent, I'll put you down | 21:56 |
| krtaylor | <krtaylor> lets move on to the agenda and we can come back to this in open discussion | 21:56 |
| krtaylor | <krtaylor> #topic Third-party CI documentation | 21:56 |
| krtaylor | <krtaylor> so this is moving along, although very slowly | 21:56 |
| krtaylor | <krtaylor> there have been a few patches merge | 21:56 |
| krtaylor | <krtaylor> but more is needed, however, I am not pushing on running-your-own as much anymore with the new openstack-ci changes coming | 21:56 |
| krtaylor | * david-lyle_afk has quit (Remote host closed the connection) | 21:56 |
| krtaylor | <krtaylor> but we will talk about that in a few minutes | 21:56 |
| krtaylor | <krtaylor> only 3 left in the queue | 21:56 |
| krtaylor | * asselin_ is back | 21:56 |
| krtaylor | <krtaylor> #link https://review.openstack.org/#/q/topic:third-party-ci-documentation,n,z | 21:56 |
| krtaylor | * ChuckC_ (~ccarlino@76-218-11-17.lightspeed.sntcca.sbcglobal.net) has joined #openstack-meeting-4 | 21:56 |
| krtaylor | * ChuckC has quit (Ping timeout: 240 seconds) | 21:56 |
| krtaylor | <ja_> #link https://review.openstack.org/#/c/149657/ looks pretty ready for >30 days. you said things have been slow. do we have a sense of the bottleneck? | 21:56 |
| krtaylor | <krtaylor> yeah, I've been trying not to annoy infra cores any more than normal and get them approved :) | 21:57 |
| krtaylor | <krtaylor> they will happen, lots of reviews from us will help refine them | 21:57 |
| krtaylor | <krtaylor> smaller chunks instead of TLDR helps too | 21:57 |
| krtaylor | <krtaylor> but, I'll see if I can beg some +2's on them | 21:57 |
| krtaylor | <krtaylor> any questions or volunteers on docs? | 21:57 |
| krtaylor | <krtaylor> this work is kinda in limbo with the new changes coming, IMO | 21:57 |
| krtaylor | <krtaylor> and that is a nice transition... | 21:57 |
| krtaylor | <krtaylor> #topic Spec for in-tree 3rd party ci solution merged | 21:57 |
| krtaylor | <krtaylor> aka openstack-ci | 21:57 |
| krtaylor | <asselin_> krtaylor, I agree we should ask them to review. These should be merged and changed again later if necessary | 21:57 |
| krtaylor | <krtaylor> agreed | 21:57 |
| krtaylor | <asselin_> krtaylor, we can send them the gerrit topic | 21:57 |
| krtaylor | * pmesserli (~pmesserli@50.56.229.5) has joined #openstack-meeting-4 | 21:57 |
| krtaylor | <krtaylor> yeah, and I think that will wrap up the documentation effort for a while, prob post summit | 21:57 |
| krtaylor | * ChuckC_ has quit (Ping timeout: 250 seconds) | 21:57 |
| krtaylor | <asselin_> yes, openstack-ci "in tree" spec merged on monday. | 21:57 |
| krtaylor | <krtaylor> asselin_, anything you want to say on the spec merge? | 21:57 |
| krtaylor | * lennyb_ (c12fa5fb@gateway/web/freenode/ip.193.47.165.251) has joined #openstack-meeting-4 | 21:57 |
| krtaylor | <krtaylor> yes, great news | 21:57 |
| krtaylor | <asselin_> gerrit topic is changing to puppet-downstream | 21:57 |
| krtaylor | <lennyb_> Hello | 21:57 |
| krtaylor | <krtaylor> hm, I thought is got changed back yesterday | 21:57 |
| *** wojdev_ has joined #openstack-meeting-4 | 21:57 | |
| krtaylor | <asselin_> #link https://review.openstack.org/#/q/topic:+downstream-puppet,n,z | 21:57 |
| krtaylor | <asselin_> I mean, downstream-puppet | 21:57 |
| krtaylor | <asselin_> krtaylor, no, we'll have one topic for the 2 related specs. | 21:57 |
| krtaylor | * krtaylor thought it was decided to be openstack-ci | 21:57 |
| krtaylor | <asselin_> the puppet module name will be that | 21:58 |
| krtaylor | <asselin_> #link http://eavesdrop.openstack.org/meetings/infra/2015/infra.2015-03-03-19.01.txt | 21:58 |
| krtaylor | <krtaylor> whatever, I don't have a strong feeling either way (not a fan of downstream) as long as it stops changing :) | 21:58 |
| krtaylor | <asselin_> krtaylor, agree. | 21:58 |
| krtaylor | * openstack (~openstack@eavesdrop.openstack.org) has joined #openstack-meeting-4 | 21:58 |
| krtaylor | * openstack has quit (Changing host) | 21:58 |
| krtaylor | * openstack (~openstack@openstack/openstack) has joined #openstack-meeting-4 | 21:58 |
| krtaylor | * ChanServ gives channel operator status to openstack | 21:58 |
| krtaylor | <asselin_> krtaylor, for reviewers, it's easier to use one topic for the 2 specs since they're related. | 21:58 |
| krtaylor | * rhe00 (~IceChat9@207.250.72.10) has joined #openstack-meeting-4 | 21:58 |
| krtaylor | <krtaylor> asselin_, yes, I was just referring to "19:27:44 <jeblair> actually the spec said "openstackci" as a topic" | 21:58 |
| krtaylor | <krtaylor> whatever, good to clarify here | 21:58 |
| krtaylor | <asselin_> so, we can start working on it now. First we'll create the new repo, and I'll add some tasks to storyboard | 21:58 |
| krtaylor | <krtaylor> anyway, what is the plan to move forward, how can we help | 21:58 |
| krtaylor | <krtaylor> ok, good | 21:58 |
| krtaylor | <asselin_> ppl can self-assign from there so we don't duplicate effort | 21:58 |
| *** sarob has joined #openstack-meeting-4 | 21:58 | |
| krtaylor | <krtaylor> agreed | 21:58 |
| *** belmoreira has joined #openstack-meeting-4 | 21:58 | |
| krtaylor | * b3rnard0 (~b3rnard0@2001:4802:7800:1:adbb:e94c:ff20:c30) has left #openstack-meeting-4 ("Textual IRC Client: www.textualapp.com") | 21:58 |
| krtaylor | <ja_> do we have a sense of when it would be usable by someone starting from scratch? | 21:58 |
| krtaylor | <krtaylor> similar to what we did with puppet-module split out | 21:58 |
| krtaylor | <asselin_> #link https://storyboard.openstack.org/#!/story/2000101 | 21:58 |
| krtaylor | * ajmiller (~ajmiller@74.202.214.170) has joined #openstack-meeting-4 | 21:58 |
| krtaylor | <ja_> e.g. would we tell someone about to start building a new one to wait 'n' weeks, or proceed now | 21:58 |
| krtaylor | <krtaylor> target liberty I would assume? | 21:58 |
| *** wojdev has quit IRC | 21:58 | |
| krtaylor | <asselin_> not much yet. I'll populate some items today. Anyone can add stuff too FYI | 21:59 |
| *** wojdev_ is now known as wojdev | 21:59 | |
| krtaylor | <asselin_> there's no target...it's a priority effort | 21:59 |
| krtaylor | <asselin_> krtaylor, but honestly, it would be great to be done by the liberty summit... | 21:59 |
| krtaylor | <krtaylor> well as with puppet-module, use us for help | 21:59 |
| krtaylor | <krtaylor> thats aggressive...but maybe doable | 21:59 |
| krtaylor | <krtaylor> asselin_, if not a hack day there would be good too | 21:59 |
| krtaylor | <asselin_> krtaylor, yes, we'll probably use that. it was very effective. | 21:59 |
| krtaylor | <krtaylor> agreed, the Friday mini sprint is very cool, nice to have everyone in one room | 21:59 |
| krtaylor | <asselin_> after we get a few patches going, we can do a sprint to review and get pieces merged | 21:59 |
| krtaylor | <krtaylor> oh, I was talking about at summit | 21:59 |
| krtaylor | <krtaylor> but a virtual sprint would be effective too | 21:59 |
| krtaylor | * evgenyf has quit (Ping timeout: 252 seconds) | 21:59 |
| *** dhellmann has joined #openstack-meeting-4 | 21:59 | |
| krtaylor | <asselin_> yes, maybe after kilo releases during the downtime. in case there are riskier changes | 21:59 |
| krtaylor | <krtaylor> agreed, this is the start of really doing something to improve the consumables | 21:59 |
| krtaylor | <krtaylor> I encourage everyone to get involved in this | 21:59 |
| krtaylor | <krtaylor> ok, anything esle you want to mention asselin_ ? | 21:59 |
| krtaylor | <asselin_> not on this topic | 21:59 |
| krtaylor | <krtaylor> ok, onward then | 21:59 |
| krtaylor | <krtaylor> #topic Repo for third party tools | 21:59 |
| krtaylor | <krtaylor> I have not made any progress on this, this week | 21:59 |
| krtaylor | <krtaylor> day job keeps getting in the way | 21:59 |
| krtaylor | <krtaylor> but, I plan on returning to this today/tomorrow | 21:59 |
| krtaylor | * galstrom_zzz is now known as galstrom | 21:59 |
| krtaylor | <krtaylor> here is the etherpad | 21:59 |
| krtaylor | <krtaylor> #link https://etherpad.openstack.org/p/third-party-ci-wg-repo | 21:59 |
| krtaylor | <krtaylor> there is some good ideas there | 21:59 |
| krtaylor | <krtaylor> I am leaning toward starting slower, after thinking about it for a while | 21:59 |
| krtaylor | <krtaylor> I think having an index of tools would be a good start | 22:00 |
| krtaylor | <krtaylor> then we can see what we have and maybe better organize | 22:00 |
| krtaylor | <patrickeast> i like that idea | 22:00 |
| krtaylor | <krtaylor> this can also come out of the system highlight discussions | 22:00 |
| krtaylor | <krtaylor> also, it gives us a way to collect up some tools and see how many we are talking about | 22:00 |
| krtaylor | <krtaylor> if we only have a few that companies are willing to share, then not worth the effort | 22:00 |
| krtaylor | <krtaylor> an index into individual github accounts would be great and serve the intent of why we are doing this in the first place | 22:00 |
| krtaylor | <krtaylor> please share any thoughts in the etherpad, I will too | 22:00 |
| krtaylor | <patrickeast> will do | 22:00 |
| krtaylor | <krtaylor> any questions on this? | 22:00 |
| krtaylor | <krtaylor> next then | 22:00 |
| krtaylor | <krtaylor> #topic What to do with monitoring dashboard | 22:00 |
| krtaylor | <krtaylor> is sweston around? | 22:00 |
| krtaylor | <krtaylor> there has been some discussion on this | 22:00 |
| krtaylor | <krtaylor> I have been thinking about it too | 22:00 |
| krtaylor | <krtaylor> it seems that what everyone wanted was to get the previous dashboard (now radar) working again | 22:00 |
| krtaylor | <patrickeast> it kind of looks like the spec hasn’t really moved forward in a while | 22:00 |
| krtaylor | <krtaylor> yeah | 22:00 |
| krtaylor | <patrickeast> oh | 22:00 |
| krtaylor | <patrickeast> i showed this at the monday meeting | 22:00 |
| krtaylor | <patrickeast> for anyone who missed that one http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack%2Fcinder&user=&timeframe=24 | 22:00 |
| krtaylor | <patrickeast> i put together a lame little dash | 22:00 |
| krtaylor | <krtaylor> not lame, excellent tool | 22:00 |
| krtaylor | <patrickeast> the original intent was that i can’t see when zuul reports a failure (only jenkins) so i wanted to monitor the event stream | 22:00 |
| krtaylor | <mmedvede> cool! | 22:00 |
| krtaylor | <patrickeast> but i let it watch all the ci accounts | 22:00 |
| krtaylor | <patrickeast> still has some serious perf issues | 22:01 |
| krtaylor | <patrickeast> needs more than a weekend of abuse to get working | 22:01 |
| krtaylor | <krtaylor> yeah, and it just needs some historical stats to be exactly what we need | 22:01 |
| krtaylor | * asselin_ likes it | 22:01 |
| krtaylor | <patrickeast> imo its a different solution than what sweston’s spec was aiming for | 22:01 |
| krtaylor | <krtaylor> patrickeast, did you ever see the radar tool working? | 22:01 |
| krtaylor | <patrickeast> no control, no stats, trends | 22:01 |
| krtaylor | <patrickeast> nope, before my time | 22:01 |
| sweston | krtaylor: yes, I am here | 22:01 |
| krtaylor | <patrickeast> https://github.com/patrick-east/scoreboard | 22:01 |
| krtaylor | <patrickeast> ^ thats the source for it if anyone wants to run it for themselves | 22:01 |
| krtaylor | <patrickeast> the ec2 thing that test one is running on sucks | 22:01 |
| krtaylor | <patrickeast> works better on a local machine with a little more network bandwidth | 22:01 |
| krtaylor | <patrickeast> easily modified to only watch your ci account and jenkins | 22:01 |
| krtaylor | * asselin_ notes his ci needs some work | 22:01 |
| krtaylor | <patrickeast> asselin: hehe its a harsh judge, if 1/5 of your tests fail it marks the ci as a fail | 22:01 |
| krtaylor | <krtaylor> radar - it was simple gauges that showed the percentages of passed, failed, skipped tests for each ci system and upstream jenkins | 22:01 |
| krtaylor | <patrickeast> ahh gotcha | 22:01 |
| krtaylor | <krtaylor> it was a set sample, I believe it was 30 days | 22:01 |
| *** eghobo has joined #openstack-meeting-4 | 22:01 | |
| krtaylor | <patrickeast> did it query gerrit or just track events? | 22:01 |
| krtaylor | <patrickeast> or both | 22:01 |
| krtaylor | <krtaylor> gathered and presented, but the urls changed for CI systems and it broke | 22:01 |
| *** wojdev has quit IRC | 22:01 | |
| krtaylor | <krtaylor> it was kinda hacky, but worked fine | 22:01 |
| krtaylor | <krtaylor> problem is that we don't have a master list of CI systems anymore | 22:01 |
| krtaylor | <patrickeast> well, with a little bit of work i can add those kind of stats to this dash if folks are interested | 22:01 |
| krtaylor | <krtaylor> except the maillist one | 22:01 |
| krtaylor | <patrickeast> i basically record the gerrit events in the db and can do whatever with em | 22:01 |
| krtaylor | <lennyb_> I made a small script to monitor last N Jenkins JObs to see if they failed. can it help you ? | 22:02 |
| krtaylor | <krtaylor> jhesketh and I discussed using his gathering backend at paris summit | 22:02 |
| krtaylor | <patrickeast> lennyb_: do you mean jenkins like the infra jenkins account posting on gerrit or a jenkins server in general? | 22:02 |
| krtaylor | * ChuckC_ (~ccarlino@15.219.162.26) has joined #openstack-meeting-4 | 22:02 |
| krtaylor | <asselin_> #link https://github.com/Triniplex/third-party-ci-dashboard | 22:02 |
| krtaylor | <asselin_> patrickeast, FYI^^ | 22:02 |
| sweston | krtaylor: can you catch me up? I have been busy with investors all day | 22:02 |
| krtaylor | <krtaylor> I think that part is fairly easy, getting the list of systems to monitor, harder | 22:02 |
| krtaylor | <lennyb_> patrickeast, Jenkins server in general | 22:02 |
| krtaylor | <patrickeast> asselin_: oh nice, thanks! | 22:02 |
| krtaylor | <asselin_> #link http://dashboard.triniplex.com/#!/ | 22:02 |
| krtaylor | <asselin_> sweston had it running here, but I don't see anything now ^^ | 22:02 |
| krtaylor | <krtaylor> actually #link https://github.com/stackforge/radar | 22:02 |
| krtaylor | <patrickeast> lennyb_: gotcha, i was looking to catch events a bit further down the chain when comments are added to gerrit, i noticed that for my ci it would sometimes report things like NOT_REGISTERED or just ERROR from zuul and jenkins would never even know it happend | 22:02 |
| krtaylor | <krtaylor> the requirement was to have a place where a developer (core) could go see if a system was behaving | 22:02 |
| krtaylor | <krtaylor> this just came up with a patch that failed on a system, but was ignored and merged anyway | 22:02 |
| krtaylor | <krtaylor> mainly because ci systems are not seen as reliable | 22:02 |
| krtaylor | <patrickeast> yea its a problem in cinder right now | 22:02 |
| krtaylor | <wznoinsk> patrickeast: listening to stream event using ssh may be more reliable | 22:02 |
| krtaylor | <krtaylor> if someone could check if it was reliable, then its report could be trusted if a patch comment showed a failure | 22:02 |
| krtaylor | <patrickeast> wznoinsk: yep, the scoreboard thing does just that with paramiko | 22:02 |
| krtaylor | <lennyb_> patrickeast, this issue we solved by searching ERROR/Traceback etc in /var/log/zuul and checking that Jenkins has done something in the last few hours ( statistics :) ) | 22:02 |
| krtaylor | * matrohon has quit (Ping timeout: 246 seconds) | 22:02 |
| krtaylor | <patrickeast> lennyb_: ahh yea that would work too | 22:02 |
| krtaylor | <krtaylor> also, radar was done without spec, not sure if this effort really requires one, but it would be good for us to unify on the work and come up with one really good solution | 22:02 |
| krtaylor | * galstrom is now known as galstrom_zzz | 22:03 |
| krtaylor | <krtaylor> so the question is - what to do with this spec? | 22:03 |
| krtaylor | <krtaylor> it seems to be gating this effort | 22:03 |
| krtaylor | <patrickeast> imo there are a few big features in the spec… it should probably be broken up | 22:03 |
| krtaylor | <krtaylor> sweston has put some fine work into it | 22:03 |
| krtaylor | <patrickeast> some of them like the ci control channel and notifications are blocking it | 22:03 |
| krtaylor | <patrickeast> when all we really need right now is a dash that shows us what is broken | 22:03 |
| krtaylor | <krtaylor> I suggested that we start with a cmd line | 22:03 |
| krtaylor | * galstrom_zzz is now known as galstrom | 22:03 |
| krtaylor | <krtaylor> oops, we are out of time | 22:03 |
| krtaylor | <krtaylor> let's continue via email | 22:03 |
| sweston | krtaylor, asselin: I just completed a move to a new office, and will be bringing the dashboard back up soon | 22:03 |
| krtaylor | <asselin_> agree, we should start with something simple. Keeps scope creep out of the spec. And put those in a new spec. | 22:03 |
| krtaylor | <krtaylor> thanks everyone, great meeting! | 22:03 |
| krtaylor | #endmeeting | 22:03 |
| *** openstack changes topic to "Zuul check queue stuck due to reboot maintenance window at one of our cloud providers - no need to recheck changes at the moment, they won't move forward." | 22:03 | |
| krtaylor | #endmeeting | 22:03 |
| openstack | Meeting ended Wed Mar 4 22:03:30 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 22:03 |
| *** ivar-lazzaro has quit IRC | 22:03 | |
| krtaylor | sweston, hehheh, that was a replay, sorry for the ping | 22:03 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/third_party/2015/third_party.2015-03-04-21.53.html | 22:03 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/third_party/2015/third_party.2015-03-04-21.53.txt | 22:03 |
| krtaylor | I am trying to get the meeting to log, openstack bot wasn't working at 1500UTC | 22:03 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/third_party/2015/third_party.2015-03-04-21.53.log.html | 22:03 |
| krtaylor | sweston, sure, let me see if I can get this meeting replay to finish properly | 22:03 |
| *** SridharRamaswamy has quit IRC | 22:03 | |
| asselin | hi | 22:05 |
| krtaylor | sorry for the pings everyone | 22:05 |
| krtaylor | trying to get the meeting to log | 22:05 |
| asselin | hi sweston...did I miss a meeting, or this is just to capture the log | 22:05 |
| asselin | ok :) | 22:05 |
| krtaylor | by replaying it here | 22:05 |
| asselin | good idea | 22:05 |
| krtaylor | it worked, but at the expense of pinging everyone | 22:06 |
| sweston | hi asselin | 22:06 |
| sweston | still a neat feature though | 22:06 |
| krtaylor | yeah, it was a good meeting, I wanted to make sure it was captured | 22:07 |
| krtaylor | we can take this to infra if you all want to discuss | 22:07 |
| asselin | ok | 22:09 |
| sweston | I don't have a plethora of time at the moment, but would like to set up something informal this week to briefly discuss the next steps for radar | 22:09 |
| sweston | anybody interested in yet another meeting this week? :-) | 22:09 |
| sweston | or am open to other suggestions, as folks have time .. don't want to put anybody out | 22:11 |
| *** ChuckC has quit IRC | 22:11 | |
| *** s3wong has quit IRC | 22:15 | |
| *** ivar-lazzaro has joined #openstack-meeting-4 | 22:16 | |
| *** ivar-laz_ has joined #openstack-meeting-4 | 22:17 | |
| *** ivar-lazzaro has quit IRC | 22:17 | |
| *** s3wong has joined #openstack-meeting-4 | 22:20 | |
| *** wojdev has joined #openstack-meeting-4 | 22:24 | |
| *** rolandchan has joined #openstack-meeting-4 | 22:43 | |
| *** ppetit_ has quit IRC | 22:49 | |
| *** bknudson has quit IRC | 22:50 | |
| *** eghobo_ has joined #openstack-meeting-4 | 22:50 | |
| *** asselin_ has joined #openstack-meeting-4 | 22:52 | |
| *** eghobo has quit IRC | 22:52 | |
| *** banix has quit IRC | 22:52 | |
| *** jecarey_ has quit IRC | 22:54 | |
| *** barrett has left #openstack-meeting-4 | 22:54 | |
| *** asselin_ has quit IRC | 22:57 | |
| *** Rockyg has joined #openstack-meeting-4 | 23:00 | |
| *** jckasper has quit IRC | 23:03 | |
| *** rolandchan has left #openstack-meeting-4 | 23:17 | |
| *** VW_ has quit IRC | 23:34 | |
| *** sarob has quit IRC | 23:36 | |
| *** Aish has quit IRC | 23:39 | |
| *** carl_baldwin has joined #openstack-meeting-4 | 23:43 | |
| *** SridharRamaswamy has joined #openstack-meeting-4 | 23:44 | |
| *** belmoreira has quit IRC | 23:44 | |
| *** jckasper has joined #openstack-meeting-4 | 23:44 | |
| *** eghobo_ has quit IRC | 23:49 | |
| *** pmesserli has quit IRC | 23:49 | |
| *** galstrom is now known as galstrom_zzz | 23:53 | |
| *** eghobo has joined #openstack-meeting-4 | 23:56 | |
| *** chuckC_ has joined #openstack-meeting-4 | 23:56 | |
| *** yamahata has joined #openstack-meeting-4 | 23:57 | |
| *** claired has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!