| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Remove support for nodepool_id https://review.openstack.org/451889 | 00:17 |
|---|---|---|
| *** jamielennox is now known as jamielennox|away | 00:25 | |
| *** jamielennox|away is now known as jamielennox | 00:31 | |
| jamielennox | jhesketh: quick question, i'm looking in 2.5 code at https://github.com/openstack-infra/zuul/blob/master/zuul/scheduler.py#L1036 | 02:58 |
| jamielennox | i'm wondering why that's inside the loop | 02:59 |
| jamielennox | if project is None how is a pipeline.source supposed to find it, regardless of how many pipelines you try? | 02:59 |
| jamielennox | in v3 it's changed in an interesting way because you don't have the project name there at all https://github.com/openstack-infra/zuul/blob/master/zuul/scheduler.py#L1036 | 03:01 |
| jhesketh | jamielennox: we need to always get the change so we can't skip the loop entirely... see the comment just above https://github.com/openstack-infra/zuul/blob/master/zuul/scheduler.py#L1025 | 03:01 |
| jamielennox | jhesketh: so the location i'm hitting is i'm getting events for things we don't handle and so the .get() returns None | 03:03 |
| jamielennox | so i'm not entirely sure on the use of project.foreign there | 03:03 |
| jamielennox | oh, right - that would be the point of foreign | 03:04 |
| jhesketh | not sure I followed what you were saying completely, but you seem to have gotten it ;p | 03:05 |
| jamielennox | jhesketh: well - maybe, does this mean that zuul is caching a local copy of everything regardless of whether it's in layout ? | 03:06 |
| jamielennox | for github we control incoming webhooks, but i attached it to gerrithub for a while there and so i would have been querying for the change of everything that happened? | 03:07 |
| jhesketh | jamielennox: I suspect so, but I'm not confident | 03:14 |
| jhesketh | I think it's necessary for cross-project deps as it's possible to need a change from a project that is otherwise not in layout.yaml | 03:15 |
| jamielennox | jhesketh: it looks like gerrit will fetch it, embed the None into a few places in the Change(), but the change gets dropped when it hits the not project check | 03:16 |
| jhesketh | jamielennox: by dropped you mean not entered into a queue? | 03:17 |
| jhesketh | the getChange I believe has the side-effect of updating the gerrit connection cache | 03:18 |
| jhesketh | it might be possible to only call getchange if the cache needs updating (ie only do it when the project doesn't exist if the change has otherwise been cached) | 03:18 |
| jamielennox | yea, the change that gets returned won't work - but it doesn't matter because it doesn't get enqueue | 03:19 |
| jamielennox | d | 03:19 |
| jamielennox | so by calling it after you make sure connection cache is valid for cross-project deps that are not in the actual layout file - got it | 03:20 |
| jamielennox | s/valid/up to date | 03:20 |
| *** yolanda has joined #zuul | 05:54 | |
| *** hashar has joined #zuul | 07:08 | |
| *** hashar is now known as hasharLunch | 10:37 | |
| *** bhavik1 has joined #zuul | 10:58 | |
| *** hasharLunch is now known as hashar | 11:19 | |
| pabelanger | morning | 12:12 |
| tobiash | hi | 12:36 |
| tobiash | I'm just asking myself if it makes sense to move the statsd reporting also into the driver architecture | 12:36 |
| tobiash | that would make it possible to export the stats also to something different like prometheus, influxdb or elasticsearch | 12:37 |
| pabelanger | tobiash: I'm hoping that once we land fedmsg support into zuul, writing apps to consume the data from fedmsg would be straightforward | 12:39 |
| pabelanger | rather then writing reporters directly into zuul | 12:40 |
| tobiash | pabelanger: so stats will also be pushed to fedmsg? | 12:40 |
| pabelanger | tobiash: I am not sure honestly, that is a jeblair questions | 12:41 |
| *** dmsimard|afk is now known as dmsimard | 12:41 | |
| pabelanger | however, if jobs results are published to fedmsg, you can also collect stats from that | 12:41 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add reporter for Federated Message Bus (fedmsg) https://review.openstack.org/426861 | 12:44 |
| *** bhavik1 has quit IRC | 13:42 | |
| *** dkranz has joined #zuul | 13:48 | |
| *** jamielennox is now known as jamielennox|away | 14:16 | |
| *** hashar has quit IRC | 14:26 | |
| *** jroll has quit IRC | 14:32 | |
| *** jroll has joined #zuul | 14:34 | |
| *** yolanda has quit IRC | 14:43 | |
| *** yolanda has joined #zuul | 14:47 | |
| jlk | Is OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'host' ([Errno -2] Name or service not known)") a normal message from a test trying to connect to a bad server? | 15:15 |
| jlk | not sure why it's trying to connect to 'host'. | 15:15 |
| jeblair | jesusaur: on second thought, i think we do want the reconfigure path to be able to schedule a merge if needed | 16:24 |
| jeblair | jesusaur: at least, that's the current behavior. | 16:30 |
| jeblair | jesusaur: maybe we can try without it. let's leave that for now and see where it gets us. | 16:32 |
| SpamapS | pabelanger: sounds like middleware to me. No thanks. I'll take drivers. | 16:42 |
| pabelanger | SpamapS: sure, but that means more things in zuul to write functional tests for | 16:43 |
| jeblair | tobiash, pabelanger, SpamapS: statsd has pluggable backends | 16:43 |
| jeblair | it's also a *really simple* protocol. i'd rather treat it as a standard protocol for stats emission and attach custom consumers to it, either via something that speaks the protocol or pluggable backends to the etsy statsd implementation. | 16:44 |
| clarkb | jeblair: I think thats afirly common, at least for influxdb and es. Not sure about prometheus. eg the tools already exist for that | 16:44 |
| pabelanger | ya, I used to do statsd + influxdb | 16:45 |
| pabelanger | worked well | 16:45 |
| SpamapS | pabelanger: yes, it does. I still prefer it. | 16:46 |
| pabelanger | not sure these days since they redid there licensing | 16:46 |
| SpamapS | unless other things accept fedmsg's directly .. a fedmsg -> lots of other things gateway in a system turns into the choke point for everything and makes my app start to be really rigid and frustratingly complex quickly. | 16:47 |
| SpamapS | Same reason I don't like RabbitMQ btw. ;) | 16:47 |
| SpamapS | pretty tough line to stay on the right side of honestly | 16:50 |
| SpamapS | AMQP 1.0 for instance basically straddles the line between enabling middleware and enabling direct comms.. and as such, only 2 things really fully speak it (qpidd, and qrouterd) | 16:51 |
| SpamapS | anyway, all this to say... I like drivers. | 16:51 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Use libyaml if possible https://review.openstack.org/451113 | 17:16 |
| *** rcarrillocruz has quit IRC | 17:33 | |
| jesusaur | jeblair: so you're ok with not scheduling the merge on reconfigure? or you think it should be re-added? | 17:33 |
| jesusaur | jeblair: I could try scheduling a merge in reEnqueueItem instead of in getLayout | 17:33 |
| jeblair | jesusaur: let's go without it for now, but keep it in mind if we see problems. | 17:35 |
| jesusaur | ok | 17:39 |
| SpamapS | pabelanger: I have to apologize. Is fedmsg just a python frontend for 0mq? | 17:42 |
| SpamapS | trying to understand it at http://www.fedmsg.com/en/latest/overview/# | 17:42 |
| clarkb | SpamapS: its more than that. Semu backend agnostic (though converging more on 0mq :( ) it also prescribes serialization and subscription mechanisms iirc | 17:44 |
| clarkb | so its a bit more complete of a tool than just emitting 0mq events on pubsub | 17:44 |
| pabelanger | SpamapS: ya, but I'm working on a mqtt backend | 18:07 |
| pabelanger | the big reason I liked it, was some of the tools fedora already build a top | 18:09 |
| pabelanger | like web apps | 18:09 |
| SpamapS | but it's just python yes? | 18:28 |
| SpamapS | Or there are other lang bindings now? | 18:28 |
| SpamapS | Also how come the Zuul user reported POST_FAILURE on https://review.openstack.org/451113 ? | 18:29 |
| SpamapS | actually that's Jenkins reporting it.. I think | 18:30 |
| SpamapS | which is weird isn't it? | 18:30 |
| SpamapS | oooo maybe we could write a TweetMQ backend for fedmsg | 18:31 |
| SpamapS | https://review.openstack.org/#/c/452219/1/specs/pike/tweetmq.rst | 18:31 |
| * SpamapS finally sees logs.o.o msg from 7 minutes ago | 18:36 | |
| pabelanger | SpamapS: ya, python | 18:38 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 19:20 |
| *** rcarrillocruz has joined #zuul | 20:49 | |
| rbergeron | spamaps: threebean (not in here, sadly) knows all the hidden magic of fedmsg, though i think pabelanger probably knows it darned well by now | 21:49 |
| rbergeron | SpamapS: also, debian also uses fedmsg | 21:49 |
| rbergeron | and clarkb: it was designed to be modular on the $whateverMQ side of things -- threebean told us it was just "i haven't documented that part but you could totally plug in a different mq" which yay pabelanger for that magic | 21:52 |
| clarkb | rbergeron: ya it was rabbit though iirc and is now mostly 0mq, which saddened me because our use of 0mq has taught us its not so great (in particular you don't get any notification of errors or disconnects nor can you even check for such states so when things break everything pretends it still happy while doing nothing) | 21:54 |
| rbergeron | lol. yeah, i guess it can be whatever | 21:55 |
| jeblair | is this error message clear enough? http://paste.openstack.org/show/605109/ | 22:08 |
| pabelanger | could the json be outputted in yaml format? | 22:09 |
| clarkb | and possibly preserve the order? | 22:09 |
| SpamapS | clarkb: you're supposed to assume that it's always broken. It's the cloud. | 22:09 |
| clarkb | SpamapS: I think that means we can all sit around and do nothing while pretending the world isn't on fire | 22:10 |
| clarkb | sounds good | 22:10 |
| SpamapS | clarkb: right. We just have to hope that at least one path through the system is not on fire sometimes. | 22:10 |
| SpamapS | pabelanger: agreed on wanting yaml there instead of json. Or at least indented whitespaced json. | 22:11 |
| SpamapS | do we not have a Mark object we could pull the original text from? | 22:12 |
| jeblair | SpamapS: we do, that's at the bottom | 22:12 |
| jeblair | it just prints the one line | 22:13 |
| jeblair | (also, what i really want to do is expand the reporter structure to do in-line comments, but i'm being a team player and not jumping down that rabbit hole until we have more basic stuff working. :) | 22:13 |
| jeblair | so i'm getting "pprint data structure dump is not great". we can probably do something about that. | 22:14 |
| SpamapS | jeblair: I've not looked at this closely, but I think we could find the mark from the deep structure with the highest line #, and show the whole project structure. | 22:14 |
| SpamapS | and yeah, more basic stuff, right | 22:14 |
| jeblair | SpamapS: i think that's what the pprint is | 22:14 |
| clarkb | jeblair: its better than not printed at all but it would help map onto what is in the file that the problem is if it was yaml | 22:14 |
| SpamapS | jeblair: indeed it's good enough for now. | 22:14 |
| jeblair | SpamapS: it's a pprint of the structure at the mark | 22:15 |
| SpamapS | throw that on the pile of neat ideas for when we're polishing things. | 22:15 |
| * SpamapS goes back to looking at tests to re-enable | 22:15 | |
| jeblair | we can probably do better than the pprint | 22:15 |
| jeblair | the fact that people don't like it is good feedback. :) | 22:15 |
| jeblair | but i was actually asking about the error message text -- is that clear enough? | 22:16 |
| SpamapS | jeblair: crystal clear. | 22:17 |
| SpamapS | which is why we went into optimization mode | 22:17 |
| SpamapS | because EENGINEER_MUST_FIND_ERROR | 22:18 |
| jeblair | SpamapS: cool, thanks :) | 22:18 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Use libyaml if possible https://review.openstack.org/451113 | 22:20 |
| * SpamapS forgot his license headers | 22:20 | |
| SpamapS | you know how they say be careful about code you write, it is a burden... | 22:20 |
| pabelanger | jeblair: I find the message clear, json to yaml is feature request :) | 22:21 |
| mordred | jeblair: fwiw - there are at least 3 +2'd on several of the secrets stack, including fungi | 22:27 |
| mordred | jeblair: but I think none of us wanted to be the one to actually pull the trigger | 22:28 |
| fungi | yeah, i didn't know who else might still be reviewing | 22:31 |
| mordred | me either | 22:31 |
| mordred | jeblair: actually, some of them have 4x+2 :) | 22:31 |
| pabelanger | Just added +2 to a few | 22:32 |
| pabelanger | after reviewing of course | 22:33 |
| jeblair | sounds like they're about ready to go in then | 22:33 |
| pabelanger | btw: we still need to land https://review.openstack.org/#/c/450894/ | 22:34 |
| pabelanger | puppet change for nodepool-launcher | 22:34 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_timer_sshkey https://review.openstack.org/450897 | 22:50 |
| fungi | my dyslexia parsed that as "time for whiskey" | 22:51 |
| fungi | even my subconscious knows it's the weekend | 22:52 |
| jesusaur | also, today would be the day to propose test_for_whiskey() (well, in an hour utc) | 22:53 |
| SpamapS | damnit | 22:57 |
| SpamapS | now it is time for whiskey | 22:57 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 23:09 |
| jesusaur | jeblair: it's sooo close | 23:10 |
| jesusaur | the only error is the first jobs in check report success instead of merge failure, but the dependent jobs correctly report merge failure | 23:11 |
| jesusaur | but since it's a race with the repo updating, I'm tempted to change the test to expect success for the first job and call it done | 23:13 |
| jeblair | jesusaur: hrm, that sounds fishy. we might want to look more closely into that next week. | 23:19 |
| jesusaur | ok | 23:21 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fully qualify project configuration names https://review.openstack.org/451970 | 23:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add hostname to TriggerEvent https://review.openstack.org/452348 | 23:26 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!