| pleia2 | zaro: can you take a look at this one? https://review.openstack.org/#/c/133844/ (should be quick, just a doc update for gearman-plugin) | 00:00 |
|---|---|---|
| *** KimJ has quit IRC | 00:01 | |
| *** shakamunyi has joined #openstack-infra | 00:02 | |
| *** amcrn has joined #openstack-infra | 00:02 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/gerritbot: Call the correct reconnect method https://review.openstack.org/133863 | 00:02 |
| *** david-lyle is now known as david-lyle_afk | 00:03 | |
| *** msabramo has joined #openstack-infra | 00:04 | |
| krotscheck | Asterisk is a thing? | 00:08 |
| *** ryanpetrello has joined #openstack-infra | 00:08 | |
| pleia2 | krotscheck: it is! | 00:09 |
| *** shakamunyi has quit IRC | 00:09 | |
| krotscheck | Weeeeird | 00:09 |
| pleia2 | https://wiki.openstack.org/wiki/Infrastructure/Conferencing | 00:09 |
| pleia2 | we sometimes use it :) | 00:09 |
| *** shakamunyi has joined #openstack-infra | 00:10 | |
| *** msabramo has quit IRC | 00:14 | |
| *** shakamunyi has quit IRC | 00:15 | |
| dims | fungi: found something that may help | 00:17 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/system-config: Switched documentation to point to storyboard. https://review.openstack.org/133864 | 00:18 |
| krotscheck | Ok, I _think_ that’s it. | 00:18 |
| * krotscheck feels like he has the most repetitive gerrit review list ever. | 00:18 | |
| pleia2 | aha, saving the best for last | 00:19 |
| pleia2 | oh nice, project_group is a thing | 00:22 |
| pleia2 | krotscheck: hm, so what is this page exactly? https://storyboard.openstack.org/#!/project_group/55 | 00:23 |
| pleia2 | I'm not sure it's really an analog to lhf | 00:23 |
| *** zns has joined #openstack-infra | 00:23 | |
| *** ryanpetrello has quit IRC | 00:23 | |
| *** Sukhdev has quit IRC | 00:24 | |
| *** Viswanath has joined #openstack-infra | 00:25 | |
| *** zns has quit IRC | 00:26 | |
| *** amitgandhinz has joined #openstack-infra | 00:27 | |
| *** achanda has quit IRC | 00:27 | |
| * pleia2 just comments in review | 00:27 | |
| *** achanda has joined #openstack-infra | 00:27 | |
| krotscheck | pleia2: That’s the aggregate status page for all activity in a project group. | 00:28 |
| krotscheck | pleia2: It seemed the closest :/ | 00:28 |
| pleia2 | krotscheck: gotcha, I'm thinking we just remove the lhf reference for now, can add it back when we get tagging going | 00:29 |
| *** Viswanath has quit IRC | 00:29 | |
| krotscheck | pleia2: Yeah, that’ll also require search terms to be persisted in the URL, which is necessary for proper history navigation. | 00:29 |
| pleia2 | I see | 00:29 |
| *** dims has quit IRC | 00:30 | |
| pleia2 | ok, I think I reviewed everything but the git-review one (jenkins is muy unhappy) and devstack-gate, which tends to take a while for tests to run | 00:30 |
| *** dims has joined #openstack-infra | 00:30 | |
| *** bdpayne_ has joined #openstack-infra | 00:31 | |
| *** amitgandhinz has quit IRC | 00:31 | |
| *** shakamunyi has joined #openstack-infra | 00:31 | |
| pleia2 | astonished that I made it through the day, but that's all I've got in me, time to give my jetlagged brain a rest for the evening (I get to unpack now, goodie!) | 00:33 |
| krotscheck | Go relax :) | 00:33 |
| *** bdpayne has quit IRC | 00:33 | |
| pleia2 | nice work on these updates | 00:34 |
| krotscheck | pleia2: Thanks. | 00:34 |
| *** AlexF has quit IRC | 00:36 | |
| *** rushiagr is now known as rushiagr_away | 00:44 | |
| *** changbl has joined #openstack-infra | 00:45 | |
| *** ihrachyshka_ has quit IRC | 00:45 | |
| *** garyh has quit IRC | 00:46 | |
| openstackgerrit | Vipul Sabhaya proposed a change to openstack-infra/project-config: Cue - Message Broker provisioning service for Openstack https://review.openstack.org/133868 | 00:50 |
| ekarlso | ey fungi u free to take a look at ^? | 00:51 |
| ekarlso | or anyone else for that matter : ) | 00:52 |
| *** dims has quit IRC | 00:52 | |
| *** melwitt has quit IRC | 00:52 | |
| *** dims has joined #openstack-infra | 00:52 | |
| krotscheck | ekarlso: Is this different from zaqar? | 00:53 |
| ekarlso | krotscheck: oh dear, it's more like trove then zaqar, control plane vs data plane .. | 00:53 |
| * ekarlso points q's @ vipul | 00:53 | |
| vipul | krotscheck: Yes, while Zaqar is a messaging API, Cue is more of a provisioning service | 00:53 |
| vipul | thx ekarlso | 00:54 |
| krotscheck | Excellent, glad to get that clarified. | 00:54 |
| *** ryanpetrello has joined #openstack-infra | 00:56 | |
| *** pc_m has quit IRC | 00:56 | |
| *** bdpayne_ has quit IRC | 00:56 | |
| *** dannywilson has quit IRC | 00:57 | |
| *** baoli has joined #openstack-infra | 00:58 | |
| *** mmaglana has quit IRC | 00:59 | |
| *** mmaglana has joined #openstack-infra | 00:59 | |
| *** JaDonG has quit IRC | 01:00 | |
| *** flaper87 has quit IRC | 01:01 | |
| arosen1 | if you're using dib with node pool i guess nodepool image-list won't show anything while the image is building? | 01:02 |
| *** flaper87 has joined #openstack-infra | 01:02 | |
| *** mmaglana has quit IRC | 01:03 | |
| *** erikmwilson has quit IRC | 01:08 | |
| *** JaDonG has joined #openstack-infra | 01:13 | |
| *** rmcall has quit IRC | 01:14 | |
| *** amcrn has quit IRC | 01:17 | |
| arosen1 | arosen | 01:19 |
| *** gokrokve has quit IRC | 01:19 | |
| *** achanda has quit IRC | 01:19 | |
| *** achanda has joined #openstack-infra | 01:20 | |
| *** dims has quit IRC | 01:21 | |
| *** dims has joined #openstack-infra | 01:22 | |
| *** mmaglana has joined #openstack-infra | 01:23 | |
| *** achanda has quit IRC | 01:25 | |
| *** wenlock has quit IRC | 01:26 | |
| *** gyee has quit IRC | 01:27 | |
| *** bhuvan has quit IRC | 01:28 | |
| *** garyh has joined #openstack-infra | 01:29 | |
| *** lttrl has joined #openstack-infra | 01:30 | |
| *** yamamoto_ has joined #openstack-infra | 01:36 | |
| *** adalbas has quit IRC | 01:38 | |
| *** weshay has joined #openstack-infra | 01:38 | |
| *** ryanpetrello has quit IRC | 01:39 | |
| *** yaguang has joined #openstack-infra | 01:43 | |
| *** yamamoto_ has quit IRC | 01:43 | |
| *** garyh has quit IRC | 01:43 | |
| openstackgerrit | Jia Dong proposed a change to openstack-infra/project-config: Modify ACL rule for create branch https://review.openstack.org/133584 | 01:47 |
| *** Ryan_Lane has quit IRC | 01:47 | |
| *** Viswanath has joined #openstack-infra | 01:47 | |
| *** Ryan_Lane has joined #openstack-infra | 01:48 | |
| *** shashankhegde has quit IRC | 01:51 | |
| *** Viswanath has quit IRC | 01:51 | |
| *** tomoe_ has joined #openstack-infra | 01:56 | |
| *** tomoe_ has quit IRC | 02:00 | |
| *** dims has quit IRC | 02:02 | |
| *** dims has joined #openstack-infra | 02:02 | |
| *** cody-somerville has quit IRC | 02:04 | |
| *** nosnos has joined #openstack-infra | 02:06 | |
| *** lttrl has quit IRC | 02:14 | |
| *** yamamoto_ has joined #openstack-infra | 02:14 | |
| *** mmaglana_ has joined #openstack-infra | 02:17 | |
| *** mmaglana has quit IRC | 02:17 | |
| *** ChuckC has quit IRC | 02:17 | |
| *** mmaglana has joined #openstack-infra | 02:20 | |
| *** mmaglana_ has quit IRC | 02:21 | |
| *** mmaglana has quit IRC | 02:21 | |
| *** mmaglana has joined #openstack-infra | 02:24 | |
| *** koolhead17 has joined #openstack-infra | 02:28 | |
| *** koolhead17 has joined #openstack-infra | 02:28 | |
| *** mmaglana has quit IRC | 02:28 | |
| *** ayoung has joined #openstack-infra | 02:29 | |
| *** garyh has joined #openstack-infra | 02:29 | |
| *** openstackgerrit has quit IRC | 02:34 | |
| *** garyh has quit IRC | 02:39 | |
| dvorak | arosen1: I assume you'd have to use nodepool dib-image-list | 02:43 |
| *** rmcall has joined #openstack-infra | 02:47 | |
| *** dims has quit IRC | 02:50 | |
| *** dims has joined #openstack-infra | 02:50 | |
| *** salv-orlando has quit IRC | 02:55 | |
| *** mattfarina has joined #openstack-infra | 02:56 | |
| *** mattfarina has quit IRC | 02:57 | |
| *** melwitt has joined #openstack-infra | 03:00 | |
| *** mmaglana has joined #openstack-infra | 03:05 | |
| *** yamahata has joined #openstack-infra | 03:07 | |
| *** jerryz has joined #openstack-infra | 03:14 | |
| *** patrickeast has quit IRC | 03:14 | |
| *** rmcall has quit IRC | 03:16 | |
| *** rmcall has joined #openstack-infra | 03:17 | |
| *** mfink has quit IRC | 03:17 | |
| *** mfink has joined #openstack-infra | 03:17 | |
| *** Viswanath has joined #openstack-infra | 03:22 | |
| *** ivar-laz_ has joined #openstack-infra | 03:23 | |
| *** ivar-laz_ has quit IRC | 03:23 | |
| *** Viswanath has quit IRC | 03:25 | |
| *** melwitt has quit IRC | 03:25 | |
| *** ivar-lazzaro has quit IRC | 03:27 | |
| *** achanda has joined #openstack-infra | 03:27 | |
| *** dkranz has quit IRC | 03:29 | |
| *** garyh has joined #openstack-infra | 03:29 | |
| *** liusheng has quit IRC | 03:31 | |
| *** liusheng has joined #openstack-infra | 03:31 | |
| *** shakamunyi has quit IRC | 03:32 | |
| *** boris-42 has quit IRC | 03:37 | |
| *** msabramo has joined #openstack-infra | 03:41 | |
| *** garyh has quit IRC | 03:44 | |
| *** nelsnelson has quit IRC | 03:49 | |
| *** Ryan_Lane has quit IRC | 03:51 | |
| *** katyafervent is now known as katyafervent_awa | 04:03 | |
| *** chandankumar has joined #openstack-infra | 04:03 | |
| *** rmcall_ has joined #openstack-infra | 04:05 | |
| *** rmcall has quit IRC | 04:05 | |
| *** rmcall_ is now known as rmcall | 04:05 | |
| *** wenlock has joined #openstack-infra | 04:06 | |
| *** henditan has joined #openstack-infra | 04:07 | |
| *** garyh has joined #openstack-infra | 04:14 | |
| *** wenlock has quit IRC | 04:15 | |
| *** liusheng has quit IRC | 04:19 | |
| *** liusheng has joined #openstack-infra | 04:20 | |
| *** garyh has quit IRC | 04:24 | |
| *** Ryan_Lane has joined #openstack-infra | 04:25 | |
| *** Ryan_Lane has joined #openstack-infra | 04:25 | |
| *** pcrews has quit IRC | 04:33 | |
| *** subscope has joined #openstack-infra | 04:36 | |
| *** zns has joined #openstack-infra | 04:41 | |
| *** bhunter71 has quit IRC | 04:46 | |
| *** BharatK has joined #openstack-infra | 04:48 | |
| *** wenlock has joined #openstack-infra | 04:49 | |
| *** shashankhegde has joined #openstack-infra | 04:50 | |
| *** koolhead17 has quit IRC | 04:53 | |
| *** henditan has quit IRC | 04:53 | |
| *** koolhead17 has joined #openstack-infra | 04:53 | |
| *** garyh has joined #openstack-infra | 04:55 | |
| *** koolhead17 has quit IRC | 04:58 | |
| *** miqui has quit IRC | 04:58 | |
| *** wenlock has quit IRC | 05:03 | |
| *** garyh has quit IRC | 05:10 | |
| *** talluri has joined #openstack-infra | 05:12 | |
| *** stevemar has quit IRC | 05:14 | |
| *** loki184 has joined #openstack-infra | 05:21 | |
| *** BharatK has quit IRC | 05:25 | |
| *** koolhead17 has joined #openstack-infra | 05:25 | |
| *** koolhead17 has joined #openstack-infra | 05:25 | |
| *** fandi has quit IRC | 05:28 | |
| *** yfried|afk has quit IRC | 05:29 | |
| *** talluri has quit IRC | 05:31 | |
| *** talluri has joined #openstack-infra | 05:31 | |
| *** msabramo has left #openstack-infra | 05:31 | |
| *** Daisy_ has joined #openstack-infra | 05:32 | |
| *** terryw has joined #openstack-infra | 05:39 | |
| *** BharatK has joined #openstack-infra | 05:41 | |
| *** otherwiseguy has quit IRC | 05:43 | |
| *** koolhead17 has quit IRC | 05:54 | |
| *** garyh has joined #openstack-infra | 05:55 | |
| *** koolhead17 has joined #openstack-infra | 05:55 | |
| *** zns has quit IRC | 05:58 | |
| *** ildikov has quit IRC | 05:58 | |
| *** koolhead17 has quit IRC | 05:59 | |
| *** yaguang has quit IRC | 06:02 | |
| *** garyh has quit IRC | 06:05 | |
| *** shashankhegde has quit IRC | 06:07 | |
| *** Vek has quit IRC | 06:08 | |
| *** xchu has joined #openstack-infra | 06:10 | |
| *** yaguang has joined #openstack-infra | 06:10 | |
| *** rmcall has quit IRC | 06:11 | |
| *** xchu has quit IRC | 06:11 | |
| *** Daisy_ has quit IRC | 06:12 | |
| *** Daisy_ has joined #openstack-infra | 06:12 | |
| *** rushiagr_away is now known as rushiagr | 06:14 | |
| *** koolhead17 has joined #openstack-infra | 06:16 | |
| *** chandankumar has quit IRC | 06:16 | |
| *** boris-42 has joined #openstack-infra | 06:21 | |
| *** fandi has joined #openstack-infra | 06:22 | |
| *** AlexF has joined #openstack-infra | 06:27 | |
| *** stevemar has joined #openstack-infra | 06:27 | |
| *** stevemar has quit IRC | 06:28 | |
| *** stevemar has joined #openstack-infra | 06:28 | |
| *** p5ntangle has joined #openstack-infra | 06:31 | |
| *** koolhead17 has quit IRC | 06:31 | |
| *** koolhead17 has joined #openstack-infra | 06:32 | |
| *** chandankumar has joined #openstack-infra | 06:35 | |
| *** talluri has quit IRC | 06:36 | |
| *** koolhead17 has quit IRC | 06:37 | |
| *** talluri has joined #openstack-infra | 06:37 | |
| *** k4n0 has joined #openstack-infra | 06:37 | |
| *** rlandy has joined #openstack-infra | 06:42 | |
| *** terryw has quit IRC | 06:44 | |
| *** vigneshvar has joined #openstack-infra | 06:44 | |
| *** vigneshvar_ has joined #openstack-infra | 06:46 | |
| *** rmcall has joined #openstack-infra | 06:46 | |
| *** _nadya_ has joined #openstack-infra | 06:46 | |
| *** koolhead17 has joined #openstack-infra | 06:49 | |
| *** vigneshvar has quit IRC | 06:49 | |
| *** garyh has joined #openstack-infra | 06:55 | |
| *** subscope has quit IRC | 06:55 | |
| *** achanda has quit IRC | 06:57 | |
| *** achanda has joined #openstack-infra | 07:01 | |
| *** chandankumar has quit IRC | 07:03 | |
| *** p5ntangle has quit IRC | 07:08 | |
| *** garyh has quit IRC | 07:09 | |
| *** AlexF has quit IRC | 07:09 | |
| *** achanda has quit IRC | 07:10 | |
| *** yolanda has joined #openstack-infra | 07:10 | |
| *** yolanda_ has joined #openstack-infra | 07:10 | |
| *** yfried|afk has joined #openstack-infra | 07:13 | |
| *** chandankumar has joined #openstack-infra | 07:17 | |
| *** chmouel has quit IRC | 07:17 | |
| *** chmouel has joined #openstack-infra | 07:19 | |
| *** AlexF has joined #openstack-infra | 07:19 | |
| *** AlexF has quit IRC | 07:20 | |
| *** chandan_kumar has joined #openstack-infra | 07:22 | |
| *** liam_ has joined #openstack-infra | 07:22 | |
| *** liam_ is now known as Guest46061 | 07:23 | |
| *** flaper87 has quit IRC | 07:23 | |
| *** chandankumar has quit IRC | 07:25 | |
| *** flaper87 has joined #openstack-infra | 07:26 | |
| *** sarob has joined #openstack-infra | 07:27 | |
| *** Guest46061 has quit IRC | 07:28 | |
| *** ildikov has joined #openstack-infra | 07:31 | |
| *** katyafervent_awa is now known as katyafervent | 07:34 | |
| *** hdd has joined #openstack-infra | 07:38 | |
| *** talluri has quit IRC | 07:38 | |
| *** HeOS has quit IRC | 07:38 | |
| *** flaper87 has quit IRC | 07:42 | |
| *** flaper87 has joined #openstack-infra | 07:42 | |
| *** chmouel has quit IRC | 07:43 | |
| *** _nadya_ has quit IRC | 07:50 | |
| *** yaguang has quit IRC | 07:51 | |
| *** dtantsur|afk is now known as dtantsur | 07:52 | |
| *** chmouel has joined #openstack-infra | 07:54 | |
| *** garyh has joined #openstack-infra | 07:55 | |
| *** e0ne has joined #openstack-infra | 07:55 | |
| *** MaxV has joined #openstack-infra | 07:56 | |
| *** bo_sh has joined #openstack-infra | 07:56 | |
| *** salv-orlando has joined #openstack-infra | 07:57 | |
| *** e0ne has quit IRC | 07:59 | |
| *** e0ne has joined #openstack-infra | 08:04 | |
| *** Vek has joined #openstack-infra | 08:04 | |
| *** yamamoto_ has quit IRC | 08:04 | |
| *** Longgeek has joined #openstack-infra | 08:04 | |
| *** yamamoto_ has joined #openstack-infra | 08:04 | |
| *** garyh has quit IRC | 08:05 | |
| *** jgjl has joined #openstack-infra | 08:05 | |
| *** yfried|afk is now known as yfried | 08:05 | |
| *** Longgeek has quit IRC | 08:06 | |
| *** Longgeek has joined #openstack-infra | 08:07 | |
| *** e0ne has quit IRC | 08:07 | |
| *** yaguang has joined #openstack-infra | 08:07 | |
| *** jgjl has quit IRC | 08:08 | |
| *** jistr has joined #openstack-infra | 08:08 | |
| *** bo_sh has left #openstack-infra | 08:09 | |
| *** yamamoto_ has quit IRC | 08:09 | |
| *** koolhead17 has quit IRC | 08:11 | |
| *** e0ne has joined #openstack-infra | 08:11 | |
| *** koolhead17 has joined #openstack-infra | 08:12 | |
| *** ildikov has quit IRC | 08:13 | |
| *** rmcall has quit IRC | 08:14 | |
| *** MaxV has quit IRC | 08:16 | |
| *** koolhead17 has quit IRC | 08:16 | |
| *** fandi has quit IRC | 08:18 | |
| *** e0ne has quit IRC | 08:20 | |
| *** rmcall has joined #openstack-infra | 08:20 | |
| *** rediskin has joined #openstack-infra | 08:20 | |
| *** camunoz has quit IRC | 08:20 | |
| *** ildikov has joined #openstack-infra | 08:25 | |
| *** jcoufal has joined #openstack-infra | 08:28 | |
| *** koolhead17 has joined #openstack-infra | 08:30 | |
| *** rmcall has quit IRC | 08:30 | |
| *** luqas has joined #openstack-infra | 08:32 | |
| *** jgjl has joined #openstack-infra | 08:33 | |
| *** stevemar has quit IRC | 08:34 | |
| *** fandi has joined #openstack-infra | 08:36 | |
| *** talluri has joined #openstack-infra | 08:39 | |
| *** mpaolino has joined #openstack-infra | 08:41 | |
| *** talluri has quit IRC | 08:43 | |
| *** talluri_ has joined #openstack-infra | 08:43 | |
| *** HeOS has joined #openstack-infra | 08:44 | |
| *** garyh has joined #openstack-infra | 08:46 | |
| *** MaxV has joined #openstack-infra | 08:53 | |
| *** xchu has joined #openstack-infra | 08:55 | |
| *** vigneshvar_ has quit IRC | 08:56 | |
| *** jpich has joined #openstack-infra | 08:56 | |
| *** arxcruz has joined #openstack-infra | 08:58 | |
| *** afazekas has joined #openstack-infra | 08:59 | |
| *** rushiagr has quit IRC | 09:01 | |
| *** rushiagr has joined #openstack-infra | 09:02 | |
| *** yaguang has quit IRC | 09:02 | |
| *** s0nea_ has joined #openstack-infra | 09:03 | |
| *** s0nea has quit IRC | 09:05 | |
| *** s0nea_ is now known as s0nea | 09:05 | |
| *** jistr has quit IRC | 09:06 | |
| *** yolanda has quit IRC | 09:07 | |
| *** yolanda_ has quit IRC | 09:07 | |
| *** garyh has quit IRC | 09:07 | |
| *** AlexF has joined #openstack-infra | 09:07 | |
| *** p5ntangle has joined #openstack-infra | 09:08 | |
| *** p5ntangl_ has joined #openstack-infra | 09:09 | |
| *** Hal_ has joined #openstack-infra | 09:09 | |
| *** derekh has joined #openstack-infra | 09:10 | |
| *** dizquierdo has joined #openstack-infra | 09:11 | |
| *** Daisy_ has quit IRC | 09:11 | |
| *** p5ntangle has quit IRC | 09:13 | |
| *** xchu has quit IRC | 09:13 | |
| *** jlibosva has joined #openstack-infra | 09:14 | |
| *** yaguang has joined #openstack-infra | 09:15 | |
| *** AlexF has quit IRC | 09:16 | |
| *** s0nea_ has joined #openstack-infra | 09:17 | |
| *** s0nea has quit IRC | 09:19 | |
| *** s0nea_ is now known as s0nea | 09:19 | |
| *** irenab has joined #openstack-infra | 09:22 | |
| *** ihrachyshka has joined #openstack-infra | 09:23 | |
| *** andreykurilin has joined #openstack-infra | 09:25 | |
| *** jistr has joined #openstack-infra | 09:26 | |
| *** amuller has joined #openstack-infra | 09:35 | |
| *** BobBall_Away is now known as BobBall | 09:35 | |
| *** mfink has quit IRC | 09:36 | |
| *** ashaeron has joined #openstack-infra | 09:36 | |
| *** vigneshvar has joined #openstack-infra | 09:37 | |
| *** talluri_ has quit IRC | 09:42 | |
| *** talluri has joined #openstack-infra | 09:43 | |
| *** dizquierdo has quit IRC | 09:48 | |
| *** p5ntangl_ has quit IRC | 09:48 | |
| *** p5ntangle has joined #openstack-infra | 09:48 | |
| *** IvanBerezovskiy has quit IRC | 09:51 | |
| *** vigneshvar has quit IRC | 09:51 | |
| *** IvanBerezovskiy has joined #openstack-infra | 09:51 | |
| *** AlexF has joined #openstack-infra | 09:52 | |
| *** AlexF has quit IRC | 09:53 | |
| *** e0ne has joined #openstack-infra | 09:53 | |
| *** garyh has joined #openstack-infra | 09:53 | |
| *** _nadya_ has joined #openstack-infra | 09:57 | |
| *** yfried has quit IRC | 09:58 | |
| *** AlexF has joined #openstack-infra | 10:00 | |
| *** vrutkovs_ has joined #openstack-infra | 10:00 | |
| *** AlexF has quit IRC | 10:01 | |
| *** yamamoto has joined #openstack-infra | 10:03 | |
| *** andreykurilin has quit IRC | 10:05 | |
| *** _nadya_ has quit IRC | 10:06 | |
| *** koolhead17 has quit IRC | 10:07 | |
| *** adalbas has joined #openstack-infra | 10:07 | |
| *** koolhead17 has joined #openstack-infra | 10:08 | |
| *** adalbas has quit IRC | 10:08 | |
| *** yamamoto has quit IRC | 10:09 | |
| *** Longgeek has quit IRC | 10:09 | |
| *** Longgeek has joined #openstack-infra | 10:10 | |
| *** achanda has joined #openstack-infra | 10:11 | |
| *** garyh has quit IRC | 10:11 | |
| *** Longgeek_ has joined #openstack-infra | 10:12 | |
| *** fandi has quit IRC | 10:12 | |
| *** koolhead17 has quit IRC | 10:12 | |
| *** vigneshvar has joined #openstack-infra | 10:13 | |
| *** achanda has quit IRC | 10:15 | |
| *** Longgeek has quit IRC | 10:15 | |
| *** krtaylor has quit IRC | 10:16 | |
| *** jp_at_hp has joined #openstack-infra | 10:17 | |
| *** yamamoto has joined #openstack-infra | 10:19 | |
| *** krtaylor has joined #openstack-infra | 10:24 | |
| *** yamamoto has quit IRC | 10:27 | |
| *** mberlin1 has joined #openstack-infra | 10:32 | |
| *** ihrachyshka has quit IRC | 10:32 | |
| *** BharatK has quit IRC | 10:36 | |
| *** dprince has joined #openstack-infra | 10:39 | |
| *** nosnos has quit IRC | 10:39 | |
| *** ihrachyshka has joined #openstack-infra | 10:41 | |
| * kashyap wonders if there's any reason why #openstack-nova is not logged here? -- http://eavesdrop.openstack.org/irclogs/ | 10:44 | |
| *** Daisy_ has joined #openstack-infra | 10:44 | |
| AJaeger | kashyap, that'S a decision of the nova team - not everybody likes to have public logging | 10:44 |
| AJaeger | please discuss with nova team | 10:44 |
| *** AlexF has joined #openstack-infra | 10:44 | |
| kashyap | AJaeger, Yes, IIRC, I briefly mentioned this with sdague a while ago, but didn't know why too. . . | 10:45 |
| kashyap | Not that this bothers me, just wondered that's all. | 10:46 |
| AJaeger | kashyap, for openstack-doc the team preferred privacy ;) | 10:47 |
| kashyap | Fair enough. | 10:47 |
| AJaeger | kashyap, my suggestion: If you like to see it for nova: Put it on the agenda of the next nova IRC meeting | 10:49 |
| AJaeger | and if there's consensus to move forward, you need to send a patch to the project config repo to enable logging | 10:49 |
| kashyap | AJaeger, Yep, noted. Thanks. | 10:50 |
| *** BharatK has joined #openstack-infra | 10:51 | |
| *** dims has quit IRC | 10:54 | |
| *** dims has joined #openstack-infra | 10:55 | |
| *** yamahata has quit IRC | 10:56 | |
| *** cdent has joined #openstack-infra | 10:56 | |
| *** Daisy_ has quit IRC | 10:58 | |
| AJaeger | SergeyLukjanov, please review https://review.openstack.org/133926 as sahara PTL... | 10:59 |
| *** reed has joined #openstack-infra | 11:02 | |
| *** nosnos has joined #openstack-infra | 11:02 | |
| *** jgjl has quit IRC | 11:07 | |
| *** pc_m has joined #openstack-infra | 11:07 | |
| *** AlexF has quit IRC | 11:08 | |
| *** pc_m has quit IRC | 11:08 | |
| *** pc_m has joined #openstack-infra | 11:08 | |
| *** loki184 has quit IRC | 11:08 | |
| *** Hal_ has quit IRC | 11:09 | |
| *** garyh has joined #openstack-infra | 11:11 | |
| *** aysyd has joined #openstack-infra | 11:14 | |
| *** Vek has quit IRC | 11:17 | |
| *** Vek has joined #openstack-infra | 11:17 | |
| *** p5ntangle has quit IRC | 11:18 | |
| *** AlexF has joined #openstack-infra | 11:21 | |
| sdague | morning folks | 11:28 |
| *** garyh has quit IRC | 11:28 | |
| AJaeger | morning, sdague ! | 11:31 |
| *** kmartin has quit IRC | 11:32 | |
| *** asselin has quit IRC | 11:40 | |
| *** yaguang has quit IRC | 11:42 | |
| *** punith has joined #openstack-infra | 11:45 | |
| *** jgjl has joined #openstack-infra | 11:48 | |
| *** p5ntangle has joined #openstack-infra | 11:51 | |
| *** AlexF has quit IRC | 11:56 | |
| chmouel | morning folks, i am having an issue with the ceilometer job in pecan http://logs.openstack.org/10/131410/2/gate/gate-pecan-tox-ceilometer-tip/b119a86/console.html | 11:56 |
| chmouel | and i am not sure where to start, surely if there was a problem with pymemcache download the ceilometer ppl would have noticed it | 11:57 |
| chmouel | this seem to happen only for pecan | 11:57 |
| sdague | chmouel: bad pymemcache release? | 11:57 |
| sdague | some times upstream libraries release badly | 11:57 |
| *** p5ntangle has quit IRC | 11:58 | |
| chmouel | it just that i would have imagined the ceilometer ppl would have noticed it (it happened since friday night i think) | 11:59 |
| *** dmsimard_away is now known as dmsimard | 11:59 | |
| *** p5ntangle has joined #openstack-infra | 11:59 | |
| *** vrutkovs_ has quit IRC | 12:00 | |
| *** talluri has quit IRC | 12:02 | |
| *** ildikov has quit IRC | 12:04 | |
| *** ZZelle has joined #openstack-infra | 12:04 | |
| *** talluri has joined #openstack-infra | 12:04 | |
| *** omrim has joined #openstack-infra | 12:05 | |
| *** baoli has quit IRC | 12:07 | |
| *** AlexF has joined #openstack-infra | 12:09 | |
| *** garyh has joined #openstack-infra | 12:11 | |
| dims | sdague: good morning, following up from python3 session, need to experiment with a fork of suds (https://review.openstack.org/#/c/127864/) - no hurries, when you get a chance is fine! | 12:12 |
| *** BharatK has left #openstack-infra | 12:12 | |
| *** mwagner_lap has quit IRC | 12:12 | |
| *** vrutkovs has joined #openstack-infra | 12:13 | |
| *** dprince has quit IRC | 12:13 | |
| *** yfried has joined #openstack-infra | 12:17 | |
| *** ildikov has joined #openstack-infra | 12:19 | |
| sdague | dims: do any projects use suds directly except oslo.vmware? | 12:20 |
| *** yfried has quit IRC | 12:20 | |
| *** garyh has quit IRC | 12:21 | |
| sdague | dims: +A | 12:21 |
| sdague | chmouel: I always expect people to notice things they don't :) | 12:21 |
| sdague | so I stop assuming that any more | 12:21 |
| *** ociuhandu has joined #openstack-infra | 12:26 | |
| chmouel | dstufft: hey there we are having an issue in http://logs.openstack.org/10/131410/2/gate/gate-pecan-tox-ceilometer-tip/b119a86/console.html the pip jobs (as tracked by cdent) is trying to download the macosx binary of pymemcache in the gates and this only happen when checking out directly the tip | 12:29 |
| chmouel | dstufft: any ideas why this may be happening? | 12:29 |
| dteselkin | Hi | 12:29 |
| dteselkin | Where can I get SCP plugin for Jenkins that is used by openstack-infra? | 12:30 |
| dstufft | chmouel: because the format for a sdist is name-version.tar.gz, the format for a "dumb" bdist is name-version-platform.tar.gz | 12:30 |
| dteselkin | It looks like "official" one (ver 1.8) doesn't have some options | 12:30 |
| dstufft | and currently *anything* is a valid version, so verison-platform looks like a valid version | 12:30 |
| dstufft | pip 1.4+ would only install it if you use --pre | 12:31 |
| dteselkin | There is no 'Copy console log' and 'Copy after failure' options | 12:31 |
| *** rushiagr is now known as rushiagr_away | 12:31 | |
| chmouel | dstufft: ok, so whos's wrong the pymemcache release or the infra job that does the pip install? | 12:32 |
| sdague | zaro: once you wander back around today, any chance in getting a bit of bootstrapping on changing gerrit ui? I'd like to see if I could manage the semantic tagging | 12:32 |
| dstufft | pymemcache should delete the bdist_dumb | 12:32 |
| dstufft | it's not useful really | 12:33 |
| dstufft | it hardcodes the entire path to the files starting with / | 12:33 |
| dstufft | if they want to do binary distribution they should use bdist_wheel and possible bdist_egg | 12:33 |
| chmouel | dstufft: that make sense thanks, i'll see if i can report a bug again pymemcache | 12:34 |
| *** cnesa has joined #openstack-infra | 12:35 | |
| *** hdd has quit IRC | 12:36 | |
| *** mpaolino has quit IRC | 12:37 | |
| *** ryanpetrello has joined #openstack-infra | 12:38 | |
| *** mrmartin has joined #openstack-infra | 12:42 | |
| dims | sdague: suds is only for vmware related code (sucks in wsdl) | 12:42 |
| *** mmaglana has quit IRC | 12:42 | |
| dims | sdague: thanks! | 12:42 |
| dims | sdague: cinder will be updated to use oslo.vmware this cycle | 12:43 |
| *** AlexF has quit IRC | 12:44 | |
| *** irenab has quit IRC | 12:45 | |
| *** talluri has quit IRC | 12:45 | |
| *** talluri has joined #openstack-infra | 12:46 | |
| *** kgiusti has joined #openstack-infra | 12:47 | |
| *** ildikov has quit IRC | 12:48 | |
| *** rkukura_ has joined #openstack-infra | 12:48 | |
| *** yfried has joined #openstack-infra | 12:49 | |
| *** adalbas has joined #openstack-infra | 12:50 | |
| *** mwagner_lap has joined #openstack-infra | 12:50 | |
| *** AlexF has joined #openstack-infra | 12:50 | |
| *** hdd has joined #openstack-infra | 12:50 | |
| *** rkukura has quit IRC | 12:50 | |
| *** rkukura_ is now known as rkukura | 12:50 | |
| *** Hal_ has joined #openstack-infra | 12:51 | |
| *** koolhead17 has joined #openstack-infra | 12:51 | |
| *** koolhead17 has joined #openstack-infra | 12:51 | |
| *** baoli has joined #openstack-infra | 12:54 | |
| *** yaguang has joined #openstack-infra | 12:55 | |
| *** weshay has quit IRC | 12:56 | |
| ekarlso | hey guys, I made the fix to https://review.openstack.org/#/c/133868 as wanted, anyone wanna ++ ? ;) | 12:56 |
| *** ildikov has joined #openstack-infra | 12:58 | |
| *** yamahata has joined #openstack-infra | 13:00 | |
| *** pelix has joined #openstack-infra | 13:01 | |
| *** krtaylor has quit IRC | 13:01 | |
| *** dprince has joined #openstack-infra | 13:01 | |
| *** yfried_ has joined #openstack-infra | 13:04 | |
| *** yfried has quit IRC | 13:04 | |
| sdague | fungi: if you are around, it looks like ER stopped running the cron job on Nov 7th | 13:04 |
| sdague | SergeyLukjanov: you might be able to look as well | 13:05 |
| *** AlexF has quit IRC | 13:06 | |
| *** afazekas has quit IRC | 13:06 | |
| *** aysyd has quit IRC | 13:06 | |
| *** henditan has joined #openstack-infra | 13:08 | |
| *** dizquierdo has joined #openstack-infra | 13:10 | |
| *** nosnos has quit IRC | 13:10 | |
| *** garyh has joined #openstack-infra | 13:11 | |
| *** AlexF has joined #openstack-infra | 13:11 | |
| *** koolhead17 has quit IRC | 13:12 | |
| *** koolhead17 has joined #openstack-infra | 13:12 | |
| mordred | chmouel: to fix it locally, we might want to add a != to pymemcache to block the bdist_dumb version name | 13:13 |
| mordred | that said ... anybody know why the pecan job there has --pre added? | 13:13 |
| mordred | (that's why we're not seeing it in the openstack gate, btw) | 13:14 |
| *** krtaylor has joined #openstack-infra | 13:15 | |
| *** bswartz has quit IRC | 13:15 | |
| *** denis_makogon has joined #openstack-infra | 13:17 | |
| *** koolhead17 has quit IRC | 13:17 | |
| mordred | AJaeger, sdague: if you get bored: https://review.openstack.org/133951 | 13:19 |
| *** aysyd has joined #openstack-infra | 13:19 | |
| sdague | mordred: +2 | 13:19 |
| sdague | mordred: hey, if you are hacking on things, any chance to look at why the cron job for elastic recheck seems to have stopped on Thursday? | 13:20 |
| sdague | I think it will require sshing into status.o.o | 13:20 |
| *** gondoi is now known as zz_gondoi | 13:24 | |
| *** dizquierdo has quit IRC | 13:27 | |
| *** AlexF has quit IRC | 13:27 | |
| *** mbacchi has joined #openstack-infra | 13:28 | |
| *** garyh has quit IRC | 13:30 | |
| *** skolekonov has quit IRC | 13:31 | |
| *** Longgeek has joined #openstack-infra | 13:33 | |
| mordred | sdague: I'm only mildly here - but let me look | 13:36 |
| sdague | ok | 13:36 |
| *** Longgeek_ has quit IRC | 13:36 | |
| mordred | sdague: hrm. any idea off the top of your head who that cron runs as? | 13:36 |
| mordred | ah - recheck | 13:36 |
| mordred | sdague: well.... there is a lock file and a bazillion copies of er_safe_run waiting on it | 13:38 |
| mordred | recheck@status:~$ ps -ef | grep flock | wc -l | 13:38 |
| mordred | 652 | 13:38 |
| *** dizquierdo has joined #openstack-infra | 13:39 | |
| mordred | sdague: is it a safe thing to do to kill er_safe_run processes? | 13:39 |
| *** hdd has quit IRC | 13:40 | |
| sdague | yeh | 13:40 |
| sdague | was there a system reboot around then? | 13:40 |
| sdague | I wonder if we stale flocked and failed alot | 13:41 |
| *** mpaolino has joined #openstack-infra | 13:41 | |
| mordred | possibly | 13:41 |
| *** weshay has joined #openstack-infra | 13:41 | |
| mordred | I'd like to suggest that we move that lock file into a /var/run dir | 13:41 |
| mordred | so that it does not persist after a reboot | 13:42 |
| mordred | however, the machien has been up for 44 days | 13:42 |
| sdague | ok, so that's not it | 13:42 |
| mordred | sdague: lock file and all elastic-recheck processes are now cleared | 13:42 |
| mordred | next cron iteration should work | 13:42 |
| sdague | cool, thanks | 13:42 |
| mordred | yay! | 13:42 |
| * mordred was helpful | 13:42 | |
| *** mmaglana has joined #openstack-infra | 13:43 | |
| *** mmaglana_ has joined #openstack-infra | 13:45 | |
| *** mmaglana has quit IRC | 13:45 | |
| *** talluri has quit IRC | 13:46 | |
| *** k4n0 has quit IRC | 13:46 | |
| *** mmaglana_ has quit IRC | 13:46 | |
| *** mmaglana has joined #openstack-infra | 13:47 | |
| *** AlexF has joined #openstack-infra | 13:47 | |
| *** mmaglana has quit IRC | 13:48 | |
| kashyap | jeblair, I'm sure you must have noticed - in Gertty, seems like line wrapping is a bit off, noticed it as I read rST documents (specs). | 13:48 |
| *** mmaglana has joined #openstack-infra | 13:48 | |
| *** pblaho has joined #openstack-infra | 13:50 | |
| *** mmaglana has quit IRC | 13:50 | |
| *** mmaglana_ has joined #openstack-infra | 13:50 | |
| *** Daisy_ has joined #openstack-infra | 13:52 | |
| *** mmaglana_ has quit IRC | 13:52 | |
| *** mmaglana has joined #openstack-infra | 13:52 | |
| mordred | kashyap: are you doing side-by-side or unified? I think there is an open feature request to do some things regarding line wrapping in side-by-side mode | 13:54 |
| *** mmaglana has quit IRC | 13:54 | |
| *** jgjl has quit IRC | 13:54 | |
| *** mmaglana has joined #openstack-infra | 13:54 | |
| kashyap | mordred, Side-by-side mode. | 13:54 |
| kashyap | Good to know; thanks. | 13:55 |
| *** mmaglana has quit IRC | 13:55 | |
| *** dims has quit IRC | 13:56 | |
| *** mmaglana has joined #openstack-infra | 13:56 | |
| *** dims has joined #openstack-infra | 13:56 | |
| *** AJaeger_ has joined #openstack-infra | 13:57 | |
| *** mmaglana has quit IRC | 13:57 | |
| *** mmaglana has joined #openstack-infra | 13:57 | |
| *** vrutkovs has quit IRC | 13:59 | |
| *** mmaglana has quit IRC | 13:59 | |
| *** mmaglana_ has joined #openstack-infra | 13:59 | |
| *** vrutkovs_ has joined #openstack-infra | 14:00 | |
| *** vrutkovs_ is now known as vrutkovs | 14:00 | |
| *** mmaglana_ has quit IRC | 14:01 | |
| *** bswartz has joined #openstack-infra | 14:01 | |
| *** mmaglana has joined #openstack-infra | 14:01 | |
| *** AlexF has quit IRC | 14:01 | |
| *** afazekas has joined #openstack-infra | 14:01 | |
| *** mattfarina has joined #openstack-infra | 14:02 | |
| *** dkranz has joined #openstack-infra | 14:03 | |
| *** mmaglana_ has joined #openstack-infra | 14:03 | |
| *** mmaglana has quit IRC | 14:03 | |
| *** yfried_ has quit IRC | 14:03 | |
| *** yfried_ has joined #openstack-infra | 14:03 | |
| *** mmaglana_ has quit IRC | 14:04 | |
| *** _nadya__ has joined #openstack-infra | 14:04 | |
| *** mmaglana has joined #openstack-infra | 14:05 | |
| *** rkukura has quit IRC | 14:05 | |
| *** afazekas has quit IRC | 14:06 | |
| *** mmaglana has quit IRC | 14:06 | |
| *** jgjl has joined #openstack-infra | 14:06 | |
| *** mmaglana has joined #openstack-infra | 14:06 | |
| *** julim has joined #openstack-infra | 14:07 | |
| *** aysyd has quit IRC | 14:07 | |
| sdague | man, jenkins05 seems pretty crochetty | 14:08 |
| *** mmaglana has quit IRC | 14:08 | |
| *** mmaglana has joined #openstack-infra | 14:08 | |
| *** bhunter71 has joined #openstack-infra | 14:10 | |
| *** mmaglana has quit IRC | 14:10 | |
| fungi | sdague: yeah, i started looking at it yesterday and elasticsearch cluster health looks fine. i'm going to restart the logstash workers and see if that helps | 14:10 |
| *** signed8bit has joined #openstack-infra | 14:10 | |
| *** mmaglana has joined #openstack-infra | 14:10 | |
| *** garyh has joined #openstack-infra | 14:11 | |
| *** mmaglana has quit IRC | 14:11 | |
| sdague | fungi: I think it was just lock backup | 14:12 |
| *** mmaglana has joined #openstack-infra | 14:12 | |
| *** IvanBerezovskiy has left #openstack-infra | 14:13 | |
| sdague | fungi: so... everything is backed up behind a non functional trove cli right now | 14:13 |
| *** mmaglana has quit IRC | 14:13 | |
| sdague | https://review.openstack.org/#/c/133930 landing could fix master | 14:13 |
| sdague | but it bounced off other things | 14:13 |
| *** mmaglana has joined #openstack-infra | 14:14 | |
| *** mmaglana_ has joined #openstack-infra | 14:16 | |
| *** mmaglana has quit IRC | 14:16 | |
| *** _nadya__ has quit IRC | 14:16 | |
| *** _nadya_ has joined #openstack-infra | 14:16 | |
| *** mmaglana_ has quit IRC | 14:17 | |
| *** dustins has joined #openstack-infra | 14:17 | |
| *** mmaglana has joined #openstack-infra | 14:17 | |
| fungi | sdague: oh, elastic-recheck. i thought you meant elasticsearch. it's pretty backed up | 14:18 |
| sdague | fungi: it's not 6 days backed up | 14:18 |
| fungi | logstash job queue of 35000 and climbing | 14:19 |
| sdague | this isn't a backup | 14:19 |
| sdague | yeh, I know, this isn't a backup | 14:19 |
| sdague | Data Last Updated: Fri Nov 07 2014 20:00:01 GMT-0500 (EST) | 14:19 |
| *** nelsnelson has joined #openstack-infra | 14:19 | |
| fungi | right, i'm saying i misread what you were initially asking about | 14:19 |
| sdague | that should always be current | 14:19 |
| *** afazekas has joined #openstack-infra | 14:19 | |
| fungi | i'd been troubleshooting the logstash queue backup and at first thought that's what you meant, but you said cron | 14:19 |
| *** mmaglana_ has joined #openstack-infra | 14:19 | |
| *** mmaglana has quit IRC | 14:19 | |
| sdague | right | 14:19 |
| *** aysyd has joined #openstack-infra | 14:19 | |
| sdague | ok, just making sure we were on the same page | 14:19 |
| *** gokrokve has joined #openstack-infra | 14:20 | |
| fungi | we are, just lagged heavily by trying to respond to things blindly while working my way through scrollback | 14:20 |
| *** rkukura has joined #openstack-infra | 14:20 | |
| *** mmaglana_ has quit IRC | 14:21 | |
| sdague | so yeh, Nov 8 1:30 UTC things stopped entirely to update, mordred tried to clear some locks, but I would have expected to see things change by now | 14:21 |
| *** ociuhandu_ has joined #openstack-infra | 14:21 | |
| sdague | oh, it looks like it just went through | 14:21 |
| *** garyh has quit IRC | 14:21 | |
| *** ociuhandu has quit IRC | 14:21 | |
| *** ociuhandu_ is now known as ociuhandu | 14:21 | |
| sdague | Data Last Updated: Wed Nov 12 2014 09:00:01 GMT-0500 (EST) | 14:21 |
| *** mmaglana has joined #openstack-infra | 14:21 | |
| sdague | ok, we should be good again, I guess the locks were the problem | 14:21 |
| *** mmaglana has quit IRC | 14:23 | |
| fungi | ahh, keen, i'll stop looking into it then | 14:23 |
| fungi | back to seeing if i can reverse the trend on the logstash queue | 14:23 |
| mordred | fungi: yah - there were 652 stuck processes :) | 14:23 |
| *** mmaglana has joined #openstack-infra | 14:23 | |
| fungi | mordred: that sounds like fun. i guess we don't wrap them in flock | 14:24 |
| sdague | fungi: can you promote - https://review.openstack.org/#/c/133930/ | 14:24 |
| fungi | sdague: looking | 14:24 |
| sdague | I just sent an email to the list about that as well | 14:24 |
| *** bcourt has joined #openstack-infra | 14:24 | |
| sdague | trove cli is broken after their keystone v3 patch | 14:24 |
| fungi | yep, on it's way in now | 14:25 |
| *** mmaglana_ has joined #openstack-infra | 14:25 | |
| *** mmaglana has quit IRC | 14:25 | |
| sdague | http://lists.openstack.org/pipermail/openstack-dev/2014-November/050269.html | 14:25 |
| *** shashankhegde has joined #openstack-infra | 14:26 | |
| *** mmaglana_ has quit IRC | 14:26 | |
| fungi | it's up front now | 14:26 |
| *** mmaglana has joined #openstack-infra | 14:26 | |
| *** mmaglana_ has joined #openstack-infra | 14:28 | |
| *** mmaglana has quit IRC | 14:28 | |
| mordred | sdague: do we not gate python-troveclient on the trove exercise? | 14:29 |
| *** mmaglana_ has quit IRC | 14:30 | |
| dims | mordred: i added a review for that - https://review.openstack.org/#/c/133955/ | 14:30 |
| *** mmaglana has joined #openstack-infra | 14:30 | |
| mordred | ah | 14:30 |
| *** Sincler has joined #openstack-infra | 14:31 | |
| *** shakamunyi has joined #openstack-infra | 14:31 | |
| dims | sdague: amrith proposed a revert - https://review.openstack.org/#/c/133958/ | 14:32 |
| *** mmaglana_ has joined #openstack-infra | 14:32 | |
| *** mmaglana has quit IRC | 14:32 | |
| *** doude_ has joined #openstack-infra | 14:33 | |
| *** AlexF has joined #openstack-infra | 14:33 | |
| *** krtaylor has quit IRC | 14:33 | |
| *** zns has joined #openstack-infra | 14:33 | |
| *** doude has quit IRC | 14:33 | |
| *** mmaglana_ has quit IRC | 14:34 | |
| *** mmaglana has joined #openstack-infra | 14:34 | |
| *** stevemar has joined #openstack-infra | 14:35 | |
| *** yfried_ is now known as yfried|afk | 14:35 | |
| *** mmaglana has quit IRC | 14:35 | |
| *** adam_g` is now known as adam_g | 14:37 | |
| *** adam_g has quit IRC | 14:37 | |
| *** adam_g has joined #openstack-infra | 14:37 | |
| *** mmaglana has joined #openstack-infra | 14:37 | |
| sdague | mordred: we do not, and honestly, we shouldn't | 14:38 |
| *** mmaglana has quit IRC | 14:39 | |
| *** miqui has joined #openstack-infra | 14:39 | |
| sdague | because that's the giant coupling problem | 14:39 |
| sdague | instead, troveclient should actually test that it works minimally on changes, which it clearly does not | 14:39 |
| *** mmaglana has joined #openstack-infra | 14:39 | |
| fungi | restarting gerritbot to see if dims's 133863 fix helps at all (which also involves upgrading to much newer gerritbot... we're still running 0.1.2 from pypi on review.o.o) | 14:39 |
| *** dkranz has quit IRC | 14:39 | |
| fungi | i'll downgrade it again if i see issues | 14:39 |
| *** openstackgerrit has joined #openstack-infra | 14:40 | |
| *** mmaglana has quit IRC | 14:40 | |
| dims | fungi: thanks | 14:41 |
| *** yfried|afk is now known as yfried_ | 14:41 | |
| *** rmcall has joined #openstack-infra | 14:41 | |
| *** mmaglana has joined #openstack-infra | 14:41 | |
| Sincler | hi, i made an upgrade of gerrit to 2.4 to 2.8, i have zuul in 2.0.0.250 and gearman in .0.0.7, but, i'm having some troubles, randomly zuul stop registering jobs, triggering gates, and then dies, in the logs doesn't appear any error, when you upgraded had this issue? | 14:41 |
| fungi | dims: no, thank you! this has been a marvellously annoying bug | 14:41 |
| *** krtaylor has joined #openstack-infra | 14:41 | |
| fungi | Sincler: you're not using the built-in gearman service in zuul? | 14:43 |
| *** mmaglana has quit IRC | 14:43 | |
| mordred | sdague: right | 14:43 |
| dims | sdague: ack, abandoned that review | 14:44 |
| mordred | sdague: however, if we gate nova changes on trove exercises, then we should probably gate python-troveclient on the same thing | 14:44 |
| *** yamamoto has joined #openstack-infra | 14:44 | |
| mordred | sdague: but I'd prefer unwinding to winding more for sure | 14:44 |
| *** thedodd has joined #openstack-infra | 14:44 | |
| sdague | well, we only do by rought accident because of the cells job | 14:44 |
| sdague | which can't run tempest | 14:44 |
| *** mmaglana has joined #openstack-infra | 14:45 | |
| sdague | which is also on the unwind list | 14:45 |
| *** xyang0 has joined #openstack-infra | 14:45 | |
| sdague | mostly, this is an accidental coupling today, I think there are a couple of ways to decouple. I think more coupling is just more pain in the future. | 14:46 |
| *** rmcall has quit IRC | 14:46 | |
| *** vhoward has joined #openstack-infra | 14:46 | |
| Sincler | fungi: how can i chek if i'm using it | 14:46 |
| kragniz | does infra manage the logging of the #openstack-* channels? | 14:46 |
| *** mmaglana has quit IRC | 14:47 | |
| *** mmaglana_ has joined #openstack-infra | 14:47 | |
| *** talluri has joined #openstack-infra | 14:47 | |
| dims | kragniz: yes, i believe so | 14:47 |
| kragniz | dims: any idea how we can get glance logs? | 14:48 |
| *** yamamoto_ has joined #openstack-infra | 14:48 | |
| sdague | it's even on my list of things that I agreed to help with - https://dague.net/openstack-goals/kilo/ | 14:48 |
| kragniz | or start them, if it's not being logged at the moment | 14:48 |
| fungi | kragniz: through patches to http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/eavesdrop.pp | 14:48 |
| kragniz | fungi: cool! | 14:48 |
| kragniz | fungi: thanks a lot | 14:48 |
| *** mmaglana_ has quit IRC | 14:48 | |
| *** AlexF has quit IRC | 14:48 | |
| *** yamamoto has quit IRC | 14:48 | |
| *** mmaglana has joined #openstack-infra | 14:48 | |
| dims | kragniz: it will show up here after that - http://eavesdrop.openstack.org/irclogs/ | 14:49 |
| kragniz | dims: yeah, that's where I was looking for them | 14:49 |
| *** chandan_kumar has quit IRC | 14:49 | |
| fungi | Sincler: it would be configured in /etc/zuul/zuul.conf (start=true in the gearman_server section) | 14:49 |
| dims | kragniz: some projects (like nova) made a conscious decision in the past not to log. (i don't know about glance) | 14:50 |
| *** mmaglana has quit IRC | 14:50 | |
| *** mmaglana has joined #openstack-infra | 14:50 | |
| *** rmcall has joined #openstack-infra | 14:50 | |
| fungi | Sincler: http://ci.openstack.org/zuul/zuul.html#zuul-conf | 14:50 |
| kragniz | dims: hmm, good point | 14:51 |
| kragniz | dims: I'll bring it up in the next glance meeting, I guess | 14:51 |
| *** dkranz has joined #openstack-infra | 14:51 | |
| fungi | kragniz: dims: right, it would need to be by consensus of the residents of the channel/leaders of the glance project | 14:51 |
| dims | kragniz: good plan | 14:51 |
| *** mmaglana has quit IRC | 14:52 | |
| *** lttrl has joined #openstack-infra | 14:52 | |
| *** mmaglana has joined #openstack-infra | 14:52 | |
| *** AlexF has joined #openstack-infra | 14:52 | |
| *** yfried_ is now known as yfried|afk | 14:52 | |
| Sincler | fungi: yes it is active | 14:53 |
| *** yfried|afk is now known as yfried_ | 14:54 | |
| *** mmaglana has quit IRC | 14:54 | |
| *** mmaglana_ has joined #openstack-infra | 14:54 | |
| fungi | Sincler: you mentioned gearman 0.0.7 so was just trying to figure out where you got that from | 14:55 |
| *** mmaglana_ has quit IRC | 14:55 | |
| *** rmcall has quit IRC | 14:55 | |
| fungi | Sincler: since if you were using zuul's built-in gearman service you wouldn't have a separate gearman version | 14:55 |
| *** mmaglana has joined #openstack-infra | 14:56 | |
| Sincler | fungi: jenkins_config::plugin { 'gearman-plugin':version => '0.0.7',} | 14:56 |
| *** p5ntangle has quit IRC | 14:56 | |
| fungi | Sincler: oh, the jenkins gearman plugin | 14:57 |
| *** talluri has quit IRC | 14:57 | |
| *** mmaglana has quit IRC | 14:57 | |
| *** mmaglana has joined #openstack-infra | 14:57 | |
| *** krtaylor has quit IRC | 14:58 | |
| fungi | Sincler: so when you say zuul "dies" you mean the zuul daemon no longer appears in the process table? | 14:58 |
| *** afazekas_ has joined #openstack-infra | 14:58 | |
| *** zns has quit IRC | 14:58 | |
| *** vigneshvar has quit IRC | 14:58 | |
| *** vrutkovs has quit IRC | 14:58 | |
| *** mmaglana has quit IRC | 14:59 | |
| *** mmaglana has joined #openstack-infra | 14:59 | |
| *** vrutkovs has joined #openstack-infra | 14:59 | |
| *** vrutkovs is now known as vrutkovs_ | 14:59 | |
| *** tonytan4ever has joined #openstack-infra | 15:00 | |
| *** afazekas has quit IRC | 15:00 | |
| *** mmaglana has quit IRC | 15:01 | |
| *** mfink has joined #openstack-infra | 15:01 | |
| *** mmaglana has joined #openstack-infra | 15:01 | |
| *** zns has joined #openstack-infra | 15:01 | |
| *** marun has joined #openstack-infra | 15:02 | |
| *** mmaglana has quit IRC | 15:02 | |
| Sincler | fungi: no, and if i try to start it again, waits almost 2 minutes to restart | 15:03 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Create repo for oslo.context https://review.openstack.org/132877 | 15:03 |
| *** mmaglana has joined #openstack-infra | 15:03 | |
| fungi | Sincler: any signs of resource starvation on the server when that happens? anything in syslog (for example, kernel out-of-memory killer sacrificed the zuul daemon)? | 15:03 |
| fungi | Sincler: under "normal" circumstances, the zuul daemon should at least log its reason for terminating | 15:04 |
| fungi | Sincler: or is it perhaps unable to append to its log because the filesystem is full? | 15:04 |
| *** mmaglana_ has joined #openstack-infra | 15:05 | |
| *** marun has quit IRC | 15:05 | |
| *** mmaglana has quit IRC | 15:05 | |
| *** zns has quit IRC | 15:05 | |
| fungi | dims: thanks--you've at least proven that tip of master gerritbot with your patch applied performs its primary function. i meant to have a review queued up to push so i could test that but got sidetracked | 15:06 |
| *** groknix has quit IRC | 15:06 | |
| *** zns has joined #openstack-infra | 15:06 | |
| *** mmaglana_ has quit IRC | 15:06 | |
| *** mmaglana has joined #openstack-infra | 15:06 | |
| dims | nice | 15:07 |
| annegentle | anyone from stackalytics know how "person-day" is calculated? I searched on the wiki and readthedocs | 15:07 |
| *** groknix has joined #openstack-infra | 15:07 | |
| annegentle | mostly a curiousity and wondering if it's useful to figure out how many writers to hire | 15:07 |
| sdague | annegentle: dice and magic eight ball? | 15:07 |
| annegentle | sdague: :) | 15:07 |
| sdague | I bet it's probably just one of the sloccount algorithms | 15:07 |
| annegentle | sdague: and wondering if it's useful to compare writer days to dev days | 15:07 |
| fungi | annegentle: in stackalytics, person-day calculates you | 15:07 |
| sdague | which I have found extremely dubious | 15:07 |
| annegentle | what's sloccount? | 15:07 |
| annegentle | I think I will submit to the calculation! | 15:08 |
| sdague | annegentle: http://www.dwheeler.com/sloccount/ | 15:08 |
| annegentle | sdague: cool thanks, reading | 15:08 |
| Sincler | fungi: the only thing that happened before, was that i send some commits with dependencies to evaluate, the one of the keeps "working" finish (almost 9 minutes after the commit was sended and it should take only 1 or 2) and then happens all, starting from zuul no longer registering jobs not triggering an not appearing in the process | 15:08 |
| *** mmaglana has quit IRC | 15:08 | |
| *** mmaglana has joined #openstack-infra | 15:08 | |
| *** dustins has quit IRC | 15:09 | |
| *** mmaglana has quit IRC | 15:10 | |
| *** mmaglana has joined #openstack-infra | 15:10 | |
| fungi | Sincler: do the jobs actually run, or do the just sit in a pending state indefinitely? | 15:10 |
| *** ildikov has quit IRC | 15:10 | |
| *** garyh has joined #openstack-infra | 15:11 | |
| *** mmaglana has quit IRC | 15:12 | |
| Sincler | fungi: only one the others not even showed in zuul neither in jenkins, | 15:12 |
| *** mmaglana has joined #openstack-infra | 15:12 | |
| *** dustins has joined #openstack-infra | 15:12 | |
| fungi | Sincler: did you also start the zuul-merger daemon? | 15:12 |
| *** ChuckC has joined #openstack-infra | 15:12 | |
| *** gokrokve has quit IRC | 15:13 | |
| *** mmaglana_ has joined #openstack-infra | 15:14 | |
| *** mmaglana has quit IRC | 15:14 | |
| Sincler | fungi: yes, everything worked fine before the upgrade | 15:14 |
| fungi | Sincler: and you updated your layout.yaml to accommodate the change in gerrit's comment stream format? | 15:15 |
| *** AlexF has quit IRC | 15:15 | |
| *** mmaglana_ has quit IRC | 15:15 | |
| *** sabeen has quit IRC | 15:15 | |
| Sincler | fungi: yes i used your layout.yaml | 15:15 |
| *** mmaglana has joined #openstack-infra | 15:16 | |
| fungi | Sincler: if you're using the same versions of zuul, jenkins, the gearman plugin and so on as you were when working with gerrit 2.4, the only thing i remember we had to change in zuul for gerrit 2.8 was the comment trigger patterns in layout.yaml | 15:16 |
| AJaeger_ | 15:17 | |
| *** mmaglana has quit IRC | 15:17 | |
| fungi | Sincler: also when you upgraded to gerrit 2.8 you needed to add the stream-events permission in the acl, but presumably you did that or zuul wouldn't be triggering anything | 15:17 |
| *** yfried_ has quit IRC | 15:17 | |
| Sincler | fungi: yes i did, | 15:17 |
| *** mmaglana has joined #openstack-infra | 15:17 | |
| fungi | Sincler: but as far as jobs failing to register, a gerrit upgrade shouldn't have affected that at all (it's not in the path between zuul's gearman service and the jenkins master) | 15:18 |
| *** dprince has quit IRC | 15:18 | |
| *** otherwiseguy has joined #openstack-infra | 15:19 | |
| fungi | Sincler: which leads me to suspect some sort of cascade problem on the zuul server. you're sure you're not running out of memory or disk space or anything there when the problem starts? | 15:19 |
| *** mmaglana has quit IRC | 15:19 | |
| *** doug-fish has left #openstack-infra | 15:19 | |
| *** mmaglana has joined #openstack-infra | 15:19 | |
| *** ryanpetrello has quit IRC | 15:19 | |
| Sincler | fungi: i had this problem only in two of the test i ran, and i have ran a lot of test | 15:20 |
| dims | anteaya: sdague: looks like when i add a blueprint, i can't set the new-project branch...long story short - https://review.openstack.org/#/c/132877/ does not show up in the new-project topic. it's a new oslo library | 15:20 |
| fungi | Sincler: oh, two of your tests cause zuul's gearman service to stop taking job registrations from your jenkins master, zuul exits and doesn't log why, not even in its debug log? that's really, really strange | 15:21 |
| *** mmaglana_ has joined #openstack-infra | 15:21 | |
| *** mmaglana has quit IRC | 15:21 | |
| *** garyh has quit IRC | 15:22 | |
| *** JayJ has joined #openstack-infra | 15:22 | |
| *** mmaglana_ has quit IRC | 15:22 | |
| Sincler | fungi: im testing all the upgrade process and in the log not appears nothing the last records is the last job | 15:22 |
| *** mmaglana has joined #openstack-infra | 15:23 | |
| fungi | Sincler: the only other thing i can think to suggest is starting the zuul daemon manually in the foreground so you can see why it exits | 15:23 |
| *** zz_dimtruck is now known as dimtruck | 15:23 | |
| *** garyh has joined #openstack-infra | 15:23 | |
| Sincler | ok | 15:23 |
| *** Vek has quit IRC | 15:23 | |
| fungi | Sincler: without knowing why the zuul daemon is silently terminating, it's hard to diagnose any of the other issues you mentioned | 15:24 |
| fungi | Sincler: if it exits because an exception is raised or because it received a signal, then it's supposed to log that | 15:24 |
| *** mmaglana has quit IRC | 15:24 | |
| *** mmaglana_ has joined #openstack-infra | 15:25 | |
| *** ryanpetrello has joined #openstack-infra | 15:25 | |
| *** skraynev has quit IRC | 15:25 | |
| mordred | http://arxiv.org/abs/1410.8001 | 15:25 |
| fungi | Sincler: so if it doesn't even tell you why it exited, then i think something is very, very wrong with your system | 15:25 |
| mordred | best. abstract. ever. | 15:25 |
| *** Daisy_ has quit IRC | 15:25 | |
| *** skraynev has joined #openstack-infra | 15:26 | |
| Sincler | fungi: ok i'll check it out, also to the performance or the memory | 15:26 |
| *** mmaglana_ has quit IRC | 15:26 | |
| *** mmaglana has joined #openstack-infra | 15:26 | |
| *** mmaglana has quit IRC | 15:28 | |
| mordred | but the essay is even better | 15:28 |
| mordred | there's math | 15:28 |
| *** mmaglana has joined #openstack-infra | 15:28 | |
| fungi | mordred: yes, hipster calculus. awesome | 15:29 |
| fungi | mordred: oh, polar trig too! | 15:30 |
| *** mmaglana has quit IRC | 15:30 | |
| *** ildikov has joined #openstack-infra | 15:30 | |
| *** mmaglana has joined #openstack-infra | 15:30 | |
| mordred | fungi: I know, right? | 15:30 |
| fungi | short but to the point | 15:30 |
| *** bauzas has joined #openstack-infra | 15:31 | |
| kragniz | mordred: the author of that is a total hipster | 15:31 |
| mordred | kragniz: yup | 15:31 |
| bauzas | sdague: hiya, FYI, I played with both reviewerin:<group> and label:Code-Review,group=nova-core and both seem buggy to me | 15:31 |
| *** mmaglana has quit IRC | 15:32 | |
| *** mmaglana_ has joined #openstack-infra | 15:32 | |
| bauzas | sdague: because reviewerin:group doesn't necessarly mean that the people voted on the current version | 15:32 |
| sdague | dims: how time critical is that? | 15:32 |
| *** cody-somerville has joined #openstack-infra | 15:32 | |
| *** gokrokve has joined #openstack-infra | 15:32 | |
| sdague | bauzas: you might be right | 15:33 |
| dims | sdague: the oslo.context? it's blocking release of oslo.log | 15:33 |
| sdague | dims: mostly I ask because jhesketh has a big refactor of all that into templates, which should ideally go first | 15:33 |
| sdague | but I need to babysit another change in with it | 15:33 |
| sdague | and I was unbreaking everything first | 15:33 |
| sdague | mostly, if we wait till tomorrow, is that ok? | 15:33 |
| bauzas | sdague: yeah, that's why I think the better way is still to use labels, but Gerrit seems buggy too, because it doesn't take the group as the option | 15:34 |
| dims | sdague: yep, works for me! | 15:34 |
| bauzas | sdague: it's a FWIW | 15:34 |
| *** dizquierdo has quit IRC | 15:34 | |
| sdague | bauzas: ok, I'll have to do some testing on it | 15:34 |
| sdague | what is the behavior you see in each case? | 15:34 |
| *** Longgeek has quit IRC | 15:35 | |
| bauzas | sdague: well, reviewerin:nova-core label:Code-Review<=-1 doesn't mean that a core expressed a disagreement, just that he's CC on the patch and someone gave a -1 | 15:35 |
| *** mmaglana has joined #openstack-infra | 15:35 | |
| sdague | bauzas: ah | 15:35 |
| bauzas | sdague: and label:Code-Review<=-1,group=nova-core shows all -1s, even non-cores | 15:36 |
| *** jgriffit1 is now known as jgriffith | 15:36 | |
| bauzas | sdague: although doc is pretty clear :) | 15:36 |
| *** mmaglana_ has quit IRC | 15:37 | |
| *** mmaglana has quit IRC | 15:37 | |
| *** mmaglana_ has joined #openstack-infra | 15:37 | |
| *** amitgandhinz has joined #openstack-infra | 15:38 | |
| *** chandankumar has joined #openstack-infra | 15:38 | |
| *** _nadya_ has quit IRC | 15:38 | |
| sdague | bauzas: yep, I'm seeing that issue | 15:39 |
| sdague | hmph | 15:39 |
| *** mmaglana_ has quit IRC | 15:39 | |
| sdague | will have to bug zaro on that | 15:39 |
| bauzas | sdague: to be clear, I'm using it for mentoring myself :) | 15:39 |
| *** mmaglana has joined #openstack-infra | 15:39 | |
| bauzas | sdague: coupled with the reviewer:self tag :) | 15:39 |
| *** mmaglana has quit IRC | 15:40 | |
| *** mmaglana has joined #openstack-infra | 15:41 | |
| sdague | fungi: also... jenkins05 seems uber slow today | 15:43 |
| *** mmaglana has quit IRC | 15:43 | |
| *** mmaglana_ has joined #openstack-infra | 15:43 | |
| fungi | sdague: oh, right, it was slow yesterday too. i meant to look into that as well | 15:43 |
| fungi | thanks for the reminder | 15:43 |
| *** mmaglana_ has quit IRC | 15:44 | |
| *** mmaglana has joined #openstack-infra | 15:45 | |
| sdague | ok, the master patch is about to merge | 15:45 |
| sdague | just uploading all the qemu logs | 15:46 |
| sdague | which, btw, is really freaking slow | 15:46 |
| *** mmaglana has quit IRC | 15:46 | |
| sdague | it's 8 minutes into log upload right now | 15:46 |
| sdague | which is usually pretty quick | 15:46 |
| *** mmaglana has joined #openstack-infra | 15:46 | |
| *** chandankumar has quit IRC | 15:47 | |
| fungi | sdague: yeah, since j05 is struggling, and it's the jenkins master process which handles log collection and uploading, it's going to be slower than usual there | 15:47 |
| *** chandankumar has joined #openstack-infra | 15:47 | |
| fungi | one of the reasons i welcome our logs in object storage overlords | 15:47 |
| *** dizquierdo has joined #openstack-infra | 15:47 | |
| *** rediskin has left #openstack-infra | 15:48 | |
| fungi | since that mechanism mostly bypasses the jenkins master and uploads artifacts directly (except that it still has to retrieve the console log) | 15:48 |
| *** mmaglana_ has joined #openstack-infra | 15:48 | |
| openstackgerrit | Aaron Rosen proposed openstack-infra/os-loganalyze: Add tr to filter for trove logs https://review.openstack.org/133981 | 15:49 |
| *** mmaglana_ has quit IRC | 15:50 | |
| openstackgerrit | Aaron Rosen proposed openstack-infra/os-loganalyze: Add tr to filter for trove logs https://review.openstack.org/133981 | 15:50 |
| *** mmaglana_ has joined #openstack-infra | 15:50 | |
| sdague | minute 11 of log upload for this job which fixes master.... | 15:50 |
| *** mmaglana has quit IRC | 15:51 | |
| *** mmaglana_ has quit IRC | 15:51 | |
| *** annegent_ has joined #openstack-infra | 15:51 | |
| fungi | once that wraps up i'll start quiescing jenkins05 for a restart | 15:52 |
| *** mmaglana has joined #openstack-infra | 15:52 | |
| afazekas_ | clarkb: ping | 15:52 |
| *** afazekas_ is now known as afazekas | 15:52 | |
| openstackgerrit | Aaron Rosen proposed openstack-infra/os-loganalyze: Add ignored Editors to .gitignore https://review.openstack.org/133982 | 15:52 |
| *** david-lyle_afk is now known as david-lyle | 15:52 | |
| *** ayoung is now known as ayoung-afk | 15:53 | |
| *** mmaglana has quit IRC | 15:53 | |
| *** mmaglana has joined #openstack-infra | 15:54 | |
| *** ihrachyshka has quit IRC | 15:55 | |
| *** rushiagr_away is now known as rushiagr | 15:56 | |
| *** mmaglana has quit IRC | 15:57 | |
| *** mmaglana has joined #openstack-infra | 15:57 | |
| *** markmcclain has joined #openstack-infra | 15:58 | |
| *** pcrews has joined #openstack-infra | 15:58 | |
| *** zz_gondoi is now known as gondoi | 15:59 | |
| *** mmaglana has quit IRC | 15:59 | |
| *** mmaglana has joined #openstack-infra | 15:59 | |
| * annegent_ waves at ttx | 15:59 | |
| *** mmaglana has quit IRC | 16:00 | |
| *** mmaglana has joined #openstack-infra | 16:01 | |
| *** mmaglana has quit IRC | 16:02 | |
| *** asselin has joined #openstack-infra | 16:02 | |
| *** thedodd has quit IRC | 16:02 | |
| *** mmaglana has joined #openstack-infra | 16:03 | |
| *** weshay has quit IRC | 16:04 | |
| *** mmaglana has quit IRC | 16:04 | |
| *** mmaglana has joined #openstack-infra | 16:05 | |
| *** rmcall has joined #openstack-infra | 16:05 | |
| *** MaxV has quit IRC | 16:07 | |
| *** talluri has joined #openstack-infra | 16:08 | |
| *** shashankhegde has quit IRC | 16:08 | |
| *** mmaglana has quit IRC | 16:08 | |
| *** mmaglana has joined #openstack-infra | 16:08 | |
| *** weshay has joined #openstack-infra | 16:09 | |
| *** mmaglana has quit IRC | 16:10 | |
| *** mmaglana has joined #openstack-infra | 16:10 | |
| *** shashankhegde has joined #openstack-infra | 16:12 | |
| *** talluri has quit IRC | 16:13 | |
| *** armax has joined #openstack-infra | 16:13 | |
| *** MaxV has joined #openstack-infra | 16:13 | |
| *** mmaglana has quit IRC | 16:13 | |
| *** mmaglana has joined #openstack-infra | 16:14 | |
| *** ashaeron has quit IRC | 16:14 | |
| *** mmaglana has quit IRC | 16:15 | |
| *** Hal_ has quit IRC | 16:15 | |
| *** mmaglana has joined #openstack-infra | 16:16 | |
| *** krtaylor has joined #openstack-infra | 16:16 | |
| *** mmaglana has quit IRC | 16:17 | |
| *** rlandy has quit IRC | 16:17 | |
| *** mmaglana has joined #openstack-infra | 16:17 | |
| *** yamahata has quit IRC | 16:19 | |
| *** amuller has quit IRC | 16:19 | |
| *** mmaglana_ has joined #openstack-infra | 16:19 | |
| *** mmaglana has quit IRC | 16:19 | |
| *** Hal has joined #openstack-infra | 16:19 | |
| *** Hal has quit IRC | 16:20 | |
| *** Hal has joined #openstack-infra | 16:20 | |
| *** Hal is now known as Guest58449 | 16:20 | |
| *** yaguang has quit IRC | 16:22 | |
| *** mmaglana_ has quit IRC | 16:22 | |
| *** mmaglana has joined #openstack-infra | 16:23 | |
| *** mmaglana_ has joined #openstack-infra | 16:24 | |
| *** mmaglana has quit IRC | 16:24 | |
| *** mmaglana_ has quit IRC | 16:25 | |
| *** mmaglana has joined #openstack-infra | 16:25 | |
| *** mmaglana has quit IRC | 16:25 | |
| *** Guest58449 has quit IRC | 16:26 | |
| openstackgerrit | Merged openstack-infra/project-config: Switched documentation to point to storyboard. https://review.openstack.org/133853 | 16:28 |
| *** sabeen has joined #openstack-infra | 16:29 | |
| *** asselin has quit IRC | 16:30 | |
| openstackgerrit | Craig Bryant proposed openstack-infra/project-config: Turn on python build jobs for monasca-api https://review.openstack.org/133995 | 16:31 |
| *** AJaeger_ has quit IRC | 16:32 | |
| *** gokrokve_ has joined #openstack-infra | 16:33 | |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: hipchat bug fix https://review.openstack.org/133997 | 16:34 |
| *** Longgeek has joined #openstack-infra | 16:35 | |
| openstackgerrit | Craig Bryant proposed openstack-infra/project-config: Turn on python build jobs for monasca-api https://review.openstack.org/133995 | 16:36 |
| *** Viswanath has joined #openstack-infra | 16:36 | |
| *** gokrokve has quit IRC | 16:36 | |
| pleia2 | good morning | 16:36 |
| sdague | fungi: you good with me kicking this in - https://review.openstack.org/#/c/132546/ ? | 16:36 |
| sdague | then baby sitting in the devstack change | 16:36 |
| *** doug-fish has joined #openstack-infra | 16:36 | |
| *** _nadya_ has joined #openstack-infra | 16:36 | |
| *** achanda has joined #openstack-infra | 16:37 | |
| *** gokrokve_ has quit IRC | 16:37 | |
| *** Viswanath has quit IRC | 16:39 | |
| fungi | sdague: in the process of reviewing it now, and can approve in moments | 16:40 |
| *** Longgeek has quit IRC | 16:40 | |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/pypi-mirror: Switched documentation to point to storyboard https://review.openstack.org/133859 | 16:40 |
| arosen1 | dvorak: thanks! | 16:41 |
| arosen1 | I didn't see that command. | 16:41 |
| sdague | fungi: sounds good | 16:41 |
| fungi | sdague: lgtm... in it goes | 16:41 |
| sdague | great | 16:42 |
| *** dtantsur is now known as dtantsur|afk | 16:43 | |
| *** dizquierdo has quit IRC | 16:44 | |
| arosen1 | fungi: what is the: zmq-publishers: tcp://localhost:8888 used for in the nodepool configuration? I don't ever see anything binding on 8888 | 16:45 |
| *** gyee has joined #openstack-infra | 16:45 | |
| *** mwagner_lap has quit IRC | 16:45 | |
| openstackgerrit | Merged openstack-infra/project-config: change client libraries to test from pypi instead of git https://review.openstack.org/132546 | 16:47 |
| *** markvan has quit IRC | 16:47 | |
| *** markvan has joined #openstack-infra | 16:47 | |
| fungi | arosen1: i think that's a placeholder and/or to support working by default if nodepool and jenkins master are the same host | 16:47 |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: adding support for custom tools plugin https://review.openstack.org/134001 | 16:47 |
| fungi | arosen1: in our production configuration, we have a list of tcp://jenkins01.openstack.org:8888 et cetera (covering all our masters) | 16:48 |
| arosen1 | fungi: if i have jenkins and nodepool on different hosts do i need to set: localhost to be the jenkins server i guess? | 16:48 |
| arosen1 | The jenkins server I have isn't binding on 8888 either though :/ | 16:48 |
| arosen1 | ah i lied it is | 16:48 |
| fungi | arosen1: yeah, and your jenkins master will listen on 8888/tcp if you have the zeromq publisher plug-in installed there | 16:48 |
| fungi | arosen1: that's how nodepool figures out when a jenkins slave transitions from idle to running to completed/offline | 16:49 |
| fungi | arosen1: so that it can map its ready/used/delete node state transitions accordingly | 16:49 |
| arosen1 | fungi: awesome thanks i understand now | 16:50 |
| fungi | arosen1: without that, you'd instead just see ready nodes pile up indefinitely in nodepool | 16:51 |
| *** afazekas has quit IRC | 16:51 | |
| fungi | (which is a good symptom to indicate you've lost zmq connectivity between nodepool/jenkins btw) | 16:51 |
| *** jlibosva has quit IRC | 16:51 | |
| arosen1 | fungi: so I think there is some problem I'm facing with zuul where I have to stop/start it every so often to get it working again. | 16:52 |
| arosen1 | I used jays scripts to setup my ci | 16:52 |
| arosen1 | I'm setting everything up again fresh without his scripts. | 16:52 |
| *** e0ne has quit IRC | 16:52 | |
| *** groknix has quit IRC | 16:52 | |
| arosen1 | I should be able to just git clone zuul; install and run merger/server with the config files right? I didn't see an install guide for it anywhere? | 16:52 |
| *** vigneshvar has joined #openstack-infra | 16:53 | |
| *** shakamunyi has quit IRC | 16:53 | |
| *** asselin has joined #openstack-infra | 16:53 | |
| *** patrickeast has joined #openstack-infra | 16:53 | |
| arosen1 | I'll add how to install it here: https://github.com/openstack-infra/zuul/blob/master/README.rst like nodepool has | 16:53 |
| fungi | arosen1: yeah, you should be able to do some variation of 'sudo pip install .' in the git clone and have a working zuul server, but you won't have configs or initscripts in place or anything. i'd recommend using the puppet module for it instead | 16:54 |
| fungi | arosen1: and you can model the install instructions off what we've got automated in puppet presumably | 16:54 |
| fungi | arosen1: if you want to document that | 16:55 |
| chmouel | mordred: so to followup on that pymemcache stuff, he just released a new version https://github.com/pinterest/pymemcache/issues/28#issuecomment-62751095 | 16:55 |
| fungi | arosen1: also, if your bug is reproducable and you get some good diagnostic details please open a story for it on storyboard.openstack.org | 16:55 |
| *** patrickeast has quit IRC | 16:55 | |
| chmouel | can i just rekick jenkins and the job will pick up the new version | 16:55 |
| chmouel | or there is osme caching issues? | 16:55 |
| chmouel | cdent: ^ | 16:56 |
| *** omrim has quit IRC | 16:57 | |
| cdent | I don't know chmouel. I would assume you can kick jenkins into a recheck but if there are mirrors involved with the pypi situation that might take some time. Are there mirrors? | 16:57 |
| *** jgjl has quit IRC | 16:57 | |
| *** ayoung-afk is now known as ayoung | 16:57 | |
| arosen1 | fungi: i wonder if this is what is causing it? http://paste.openstack.org/show/132437/ this is the only trace i see in my zuul logs | 16:57 |
| arosen1 | I wonder if it isn't able to recover from that? | 16:57 |
| *** arxcruz has quit IRC | 16:57 | |
| fungi | cdent: chmouel: there are mirrors, updated every 5 minutes unless something is wrong with them | 16:57 |
| chmouel | cdent: yeah the question asking to infra :) this was just to let you know of the status :) | 16:57 |
| cdent | 5 minutes? that's nice and fast. | 16:58 |
| chmouel | fungi: cool, perfect, thanks! | 16:58 |
| cdent | chmouel: cool, perfect, thanks! ;) | 16:58 |
| fungi | chmouel: if you can find it in http://pypi.dfw.openstack.org/simple/pymemcache/ for example, then you should be cool | 16:58 |
| * cdent sees new version | 16:58 | |
| chmouel | fungi: oh yeah that sounds about right! the bad macosx version is there too | 16:59 |
| chmouel | and exe for what is worth | 16:59 |
| chmouel | maybe that something we can check when doing the mirroring ? | 16:59 |
| fungi | yep, we run bandersnatch and mirror everything from pypi these days | 16:59 |
| *** ildikov has quit IRC | 16:59 | |
| chmouel | ah gotcha | 16:59 |
| fungi | with separate pypi mirror servers in each provider/region | 16:59 |
| *** mfink__ has joined #openstack-infra | 16:59 | |
| chmouel | that one is very fancy, pymemcache-0.9.macosx-10.8-intel.exe i wonder if that works with that new opensource .net for unices | 17:00 |
| *** erlon1 has quit IRC | 17:00 | |
| *** achanda has quit IRC | 17:00 | |
| *** asselin has quit IRC | 17:02 | |
| *** patrickeast has joined #openstack-infra | 17:02 | |
| *** mfink has quit IRC | 17:02 | |
| *** mattfarina has quit IRC | 17:03 | |
| openstackgerrit | Aaron Rosen proposed openstack-infra/nodepool: Update Readme config -> system-config repo https://review.openstack.org/134008 | 17:03 |
| *** mmaglana has joined #openstack-infra | 17:04 | |
| *** MaxV has quit IRC | 17:04 | |
| *** mattfarina has joined #openstack-infra | 17:04 | |
| *** groknix has joined #openstack-infra | 17:05 | |
| *** talluri has joined #openstack-infra | 17:05 | |
| *** garyh has quit IRC | 17:06 | |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: hipchat bug fix https://review.openstack.org/133997 | 17:07 |
| *** groknix has quit IRC | 17:08 | |
| tchaypo | https://review.openstack.org/#/c/129533/ now has a +2 and passing tests | 17:08 |
| *** groknix has joined #openstack-infra | 17:08 | |
| *** _shaps_ has joined #openstack-infra | 17:08 | |
| tchaypo | thanks to the patience of AJaeger | 17:09 |
| *** chandankumar has quit IRC | 17:10 | |
| *** asselin has joined #openstack-infra | 17:11 | |
| *** gokrokve has joined #openstack-infra | 17:12 | |
| *** mmaglana has quit IRC | 17:13 | |
| *** marun has joined #openstack-infra | 17:15 | |
| fungi | tchaypo: what's the story with stackforge projects managed by an openstack core reviewer group? this looks like it would be the first stackforge project using the tripleo-incubator acl. why isn't it an openstack project? | 17:17 |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: adding support for the custom tools plugin https://review.openstack.org/134011 | 17:17 |
| *** vrutkovs_ has quit IRC | 17:17 | |
| *** shashankhegde has quit IRC | 17:18 | |
| *** wenlock has joined #openstack-infra | 17:18 | |
| HenryG | Hi, is the slave_db still installed on gate nodes for running unit tests? | 17:18 |
| *** cdent has quit IRC | 17:18 | |
| *** henditan has quit IRC | 17:19 | |
| HenryG | This log seems to indicate it is not. http://logs.openstack.org/36/124136/9/check/gate-neutron-python27/372bb18/console.html | 17:19 |
| *** thedodd has joined #openstack-infra | 17:19 | |
| *** Vek has joined #openstack-infra | 17:19 | |
| fungi | HenryG: at this point we make it possible for jobs to add and delete their own mysql and postgres databases | 17:19 |
| openstackgerrit | Craig Bryant proposed openstack-infra/project-config: Turn on python build jobs for monasca-api https://review.openstack.org/133995 | 17:20 |
| HenryG | fungi: OK, and how does a job do that? Is there an example? | 17:20 |
| tchaypo | fungi: chalk it up to me being confused, I guess. I couldn’t decide what I wanted to I decided to whack something in and change it later if it becomes clear that it’s wrong | 17:20 |
| *** ivar-lazzaro has joined #openstack-infra | 17:20 | |
| fungi | HenryG: it's recommended you have per-test databases too so as to avoid concurrency issues | 17:21 |
| fungi | HenryG: i'd suggest looking at how nova engineered their database interactions in unit tests | 17:21 |
| *** vigneshvar has quit IRC | 17:21 | |
| *** koolhead17 has joined #openstack-infra | 17:21 | |
| fungi | tchaypo: okay, cool. i'll leave comments | 17:21 |
| tchaypo | spamaps: do you have an opinion about whether prep-source-repos should be stackforge/ or openstack/? If the latter it sounds like maybe we should create other groups for the acts? | 17:21 |
| SpamapS | tchaypo: so if we're going to use it in our CI system, I think openstack/ | 17:22 |
| HenryG | fungi: Ah, so the DB backends and DB *user* is installed? Just need to create our own databases? | 17:22 |
| *** _nadya_ has quit IRC | 17:22 | |
| tchaypo | i shall do the needful | 17:23 |
| clarkb | so you know zuul does this right... | 17:23 |
| *** dannywilson has joined #openstack-infra | 17:23 | |
| * clarkb goes back to being on vacation | 17:23 | |
| SpamapS | tchaypo: Sorry I haven't put much thought into it until now. I think it's another nice thing we've produced that others can consume like os-cloud-config .. and we certainly want to keep using it. | 17:23 |
| *** mmaglana has joined #openstack-infra | 17:23 | |
| fungi | HenryG: that's the way i understand it, yeah | 17:24 |
| greghaynes | clarkb: ours is a little diferent im pretty sure - we can stack non-dependent patches | 17:24 |
| clarkb | greghaynes zuul could do that too | 17:24 |
| greghaynes | if this is the tool im thinking of... | 17:24 |
| clarkb | its just git refs in the end | 17:24 |
| fungi | tchaypo: i suggest we make that openstack/ instead in that case, and you'll want to propose a change to openstack/governance to add it to reference/programs.yaml and have SpamapS +1 it | 17:25 |
| clarkb | openstack*/ doesnt need two tools that do this | 17:25 |
| clarkb | fungi ^ | 17:25 |
| fungi | (or at least have him comment favorably on it if he can't +1 there) | 17:25 |
| mordred | clarkb: but it does need one that meets their use Case | 17:25 |
| mordred | zuul does not, currently | 17:25 |
| clarkb | mordred sure but it could if we worked together on the common tool | 17:26 |
| mordred | ++ | 17:26 |
| *** chandankumar has joined #openstack-infra | 17:26 | |
| tchaypo | +1 for having just one tool that does this | 17:26 |
| tchaypo | our use case at the moment is on the ci machines; we often have patches that are in-flight that we need in order to build out the cloud | 17:27 |
| SpamapS | Could zuul be expanded to do this, reasonably? | 17:27 |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: add support to git for changelog against branch https://review.openstack.org/134014 | 17:27 |
| tchaypo | prep-source-repos lets us give a list of in-flight changes and plop them on top of trunk | 17:27 |
| SpamapS | So not just the change being tested in the pipeline.. | 17:27 |
| clarkb | tchaypo that is exactly what zuul does | 17:27 |
| SpamapS | these are changes that are usually reverts of things where upstream broke stuff | 17:28 |
| greghaynes | as is I thought zuul sets a set of repos each to a specific ref? | 17:28 |
| tchaypo | is it something I can easily drive by hand on a machine that in a brand new region? Can zuul be prodded into exporting the DIB_REPO* variables so that DIB will use those repos? | 17:28 |
| clarkb | right now it only understands refs as provided by the zuul server but having an arbitrary set should be trivial to add | 17:28 |
| clarkb | because the git stuff is already there | 17:28 |
| tchaypo | yeah, it’s quite important that we be able to pull in arbitrary sets of changes. | 17:29 |
| *** vigneshvar has joined #openstack-infra | 17:29 | |
| mordred | greghaynes: right, but it creates those refs from in flight changes | 17:29 |
| *** mmaglana has quit IRC | 17:29 | |
| SpamapS | Note that we may not be alone in this. Other projects who want to run independent functional tests in their gates may also want this. | 17:29 |
| * tchaypo goes away to read about zuul | 17:29 | |
| tchaypo | yeah, I was talking to someone at summit who has a very similar use case | 17:29 |
| clarkb | SpamapS I am just saying openstack has this tool. if it needs a new feature please add it instead of making new project | 17:30 |
| mordred | the general topic of "given a set of unrelated changes, merge them each into their repos and tell me the refs" is what zuul does | 17:30 |
| tchaypo | https://github.com/Netflix/zuul doesn’t seem to be the right zuul | 17:30 |
| greghaynes | tchaypo: http://git.openstack.org/cgit/openstack-infra/zuul/tree/README.rst | 17:30 |
| *** dangers_away is now known as dangers | 17:31 | |
| *** amcrn has joined #openstack-infra | 17:31 | |
| tchaypo | "Zuul is a trunk gating system developed for the OpenStack Project.” - trunk gating isn’t what I’m trying to do | 17:31 |
| tchaypo | *Reads other docs* | 17:32 |
| SpamapS | clarkb: and we are going "hm, you may be right" and working out what it might be. So thanks. :) | 17:32 |
| pleia2 | tchaypo: also see http://ci.openstack.org/zuul.html (has links to configuration, full manual, etc) | 17:32 |
| SpamapS | tchaypo: Yes, it is what you're trying to do. :) | 17:32 |
| tchaypo | ah, i was searching http://docs.openstack.org/ :) | 17:32 |
| tchaypo | thanks pleia2 | 17:32 |
| *** vigneshvar_ has joined #openstack-infra | 17:32 | |
| SpamapS | tchaypo: you're just trying to do it with trunk+stuff, and trying to duplicate said gate on your local machine :) | 17:33 |
| arosen1 | fungi: why does nodepool require you to set: NODEPOOL_SSH_KEY? Why isn't that just done via /etc/nodepool/nodepool.yaml ? | 17:33 |
| fungi | so while i personally would love to see patches to zuul-cloner to support that use case, i also don't want to get into the business of being responsible for telling people where and how they should be contributing so if they want a separate project for that i'm not going to stand in their way (and it apparently has been a separate project for quite a while, this is just making it more accessible) | 17:33 |
| tchaypo | SpamapS: yeah, close enough | 17:33 |
| clarkb | fungi generally agree except they want to use this in the gate | 17:34 |
| tchaypo | The main thing I’m looking for is how hard it is to set up zuul, give it a list of change sets, and get a set of DIB_REPO* variables from it | 17:34 |
| clarkb | fungi and we really dont need two tools with two sets of bugs doing the same thing in the gate | 17:34 |
| tchaypo | clarkb: no, I want to use this to stand up a region | 17:34 |
| *** vigneshvar has quit IRC | 17:35 | |
| SpamapS | fungi: I think it may save a lot of trouble to suggest that we may want to explore adding this to zuul rather than a new project. | 17:35 |
| tchaypo | spamaps’ comments are that that’s functionally similar to putting things through the gate, but I’m not intending to get this involved in the actual gate at all | 17:35 |
| *** _nadya_ has joined #openstack-infra | 17:35 | |
| *** sunil_ has quit IRC | 17:35 | |
| clarkb | tchaypo I see | 17:36 |
| SpamapS | tchaypo: perhaps we're doing this wrong then. If we just need to maintain a divergent tree in our git repo, we should just do so in a branch and use git merges. | 17:36 |
| SpamapS | tchaypo: why would we need a tool for that at all? | 17:36 |
| clarkb | in any case zuul cloner essentially takes a tip git ref and applies to all projects | 17:37 |
| SpamapS | tchaypo: I had thought this was for us to be able to stop adding little revert commands to tripleo-ci and duplicating said commands on local devtest runs. You seem to be saying it is a more visceral thing.. happening just to build a cloud in the moment. ? | 17:37 |
| *** bhuvan has joined #openstack-infra | 17:37 | |
| tchaypo | yes, as far as i know this is only used at present to build the ci regions | 17:37 |
| clarkb | what I had in mind is a list of refs that are applied in order as an extension to that as it should be backward compat if done that way | 17:37 |
| greghaynes | yea, looking at http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/lib/cloner.py its pretty close to functionally identical | 17:38 |
| *** jistr has quit IRC | 17:38 | |
| *** gondoi is now known as zz_gondoi | 17:39 | |
| *** _nadya_ has quit IRC | 17:39 | |
| SpamapS | tchaypo: ok, so IMO you don't need a tool for this. clone upstream. branch master into something like 'tripleo-cd' and merge master into tripleo-cd as needed. | 17:39 |
| fungi | arosen1: i don't see it being exported by the calling environment in the nodepool initscript. what do you mean specifically by "required"? | 17:39 |
| SpamapS | tchaypo: That is a one-time-ever thing, so I'm dubious that you need a tool to do that. | 17:39 |
| tchaypo | it’s not a one-time-ever thing | 17:40 |
| *** dannywilson has quit IRC | 17:40 | |
| tchaypo | it’s one-time-per-patchset | 17:40 |
| *** zz_gondoi is now known as gondoi | 17:40 | |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: hipchat bug fix https://review.openstack.org/133997 | 17:40 |
| SpamapS | tchaypo: if you have a patch you need that isn't in master, you apply it in 'tripleo-cd' | 17:41 |
| tchaypo | some of the patches we wanted for hp2 went through ~36 patchests before landing | 17:41 |
| fungi | arosen1: i see it mentioned in the readme, but nodepool itself doesn't appear to use it | 17:41 |
| arosen1 | fungi: if I don't export it before running nodepoold -d then I don't believe my key gets propagated to the instances? https://github.com/openstack-infra/nodepool/blame/master/README.rst#L65 | 17:41 |
| *** amcrn has quit IRC | 17:41 | |
| SpamapS | tchaypo: ah so the issue is that we want to keep rebasing off those inflight things? | 17:41 |
| tchaypo | i *think* that in those cases, each time there’s a new patchset, tripleo-cd needs to be reverted back to master, then the relevant patchsets have to be pulled in again | 17:41 |
| SpamapS | tchaypo: well not reverted.. rebased. :) | 17:41 |
| fungi | arosen1: er, you'd use something like puppet to install a public key into authorized_keys on the nodes | 17:42 |
| tchaypo | yeah, an interactive rebase could probably do it - remove the old commit, leave in the new commit | 17:42 |
| tchaypo | tricky when there’s 9 different patchsets being applied, but doable | 17:43 |
| tchaypo | in any case, it sounds like zuul does something similar. I’m down with using one tool per job, so I’m looking at whether that’s going to work | 17:43 |
| tchaypo | (adam_g | 17:43 |
| tchaypo | and lifeless have written more of the script than I have, they might have other comments to add) | 17:43 |
| SpamapS | tchaypo: so I see where a tool is desired, because of a desire to basically keep up with multiple branches. | 17:44 |
| SpamapS | tchaypo: my thought was that we were more doing the thing where we pull an inflight patch in, and it fixes us, and we just never pull it again. | 17:44 |
| tchaypo | reading http://ci.openstack.org/zuul/merger.html it sounds like the zuul merger is rather more intensive - it sounds like it runs as a daemon, receives events, does the merging automatically | 17:44 |
| fungi | arosen1: aha, we reference that in openstack-infra/project-config:nodepool/scripts/prepare_node.sh | 17:45 |
| zigo | Hey, I quickly did a package for Zuul, as I heard some were interested by it. It's in: git://git.debian.org/openstack/zuul.git | 17:45 |
| zigo | Will it help -infra ? | 17:45 |
| clarkb | tchaypo you want cloner not merger. its a utility for fetching and applying refs | 17:45 |
| tchaypo | oh | 17:45 |
| * tchaypo skips down some docs | 17:45 | |
| fungi | zigo: probably won't help us directly. we continuously deploy zuul via pip install from the tip of its master branch | 17:45 |
| zigo | fungi: Ok. | 17:45 |
| clarkb | zigo there is a zuul package repo on our gerrit too iirc | 17:46 |
| fungi | zigo: though having it packaged in debian is awesome anyway | 17:46 |
| clarkb | fungi or was that jjb? | 17:46 |
| zigo | fungi: I didn't plan on uploading it, though I can if you think it may help. | 17:46 |
| zigo | In such case, just let me know. | 17:46 |
| *** jpich has quit IRC | 17:46 | |
| clarkb | http://git.openstack.org/cgit/openstack-infra/zuul-packaging/ | 17:47 |
| fungi | clarkb: zigo: yeah, not sure how far pabelanger got with that effort | 17:47 |
| clarkb | (its empty) | 17:47 |
| fungi | ahh, yep. i think it was a placeholder while he began work | 17:48 |
| tchaypo | reading the page about zuul-cloner, I see that I can specify *a* zuul_change or *a* zuul ref - it’s not clear to me how I could give it a list of 9 unrelated changes to merge | 17:48 |
| fungi | oh, not empty... http://git.openstack.org/cgit/openstack-infra/zuul-packaging/tree/debian?h=debian/sid | 17:48 |
| fungi | clarkb: remember the master branch contains nothing but a readme in the model we were evaluating | 17:48 |
| clarkb | oh right | 17:49 |
| clarkb | tchaypo that would be the new feature | 17:49 |
| tchaypo | ah | 17:49 |
| pabelanger | fungi, clarkb I posted some patches a while ago, however there really isn't the group of people to review / merge them | 17:49 |
| tchaypo | would it even be worth adding it as a new feature? Would simply running zuul_cloner once for each change be sufficient? | 17:49 |
| tchaypo | or would each run wipe out the change from the previous run? | 17:50 |
| fungi | pabelanger: yeah, i think we were hoping more people interested in packaging things for various distros would engage there | 17:50 |
| tchaypo | scusi, laundy time | 17:50 |
| clarkb | tchaypo it would reset and wipe out | 17:50 |
| pabelanger | https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul-packaging,n,z | 17:50 |
| pabelanger | fungi, nice | 17:50 |
| *** armax has quit IRC | 17:50 | |
| pabelanger | I'm still interested in seeing it grow, I just haven't had time to focus on it ATM. Due to another contract I am working on | 17:51 |
| pabelanger | But, should have some time in the next month or so | 17:51 |
| fungi | clarkb: tchaypo: given zuul-cloner works on named refs, you could just add a lightweight tag with some common name pointing at the correct ref in each of the various repos and then have it integrate those, right? | 17:52 |
| sdague | fungi: is the NOT_REGISTERED here transient - https://review.openstack.org/#/c/133958/ ? | 17:52 |
| fungi | clarkb: tchaypo: oh, i guess what's missing is a separate mechanism to stack up multiple changes in a single repo a la zuul's dependent pipeline operation | 17:53 |
| clarkb | fungi ya | 17:53 |
| fungi | sdague: probably. i'll confirm | 17:53 |
| *** vigneshvar_ has quit IRC | 17:55 | |
| *** HeOS has quit IRC | 17:55 | |
| tchaypo | How about I spend a bit of time tonight playing with zuul_cloner to see if I can figure out what would be needed to do to it | 17:55 |
| *** chandankumar has quit IRC | 17:55 | |
| fungi | sdague: yeah, transient. i see both those jobs registered for both relevant node types now | 17:55 |
| sdague | ok | 17:55 |
| *** shakamunyi has joined #openstack-infra | 17:56 | |
| tchaypo | Then I'll comment on the change (or update the read me for the project?) to mention that this overlaps with zuul_cloner and should go away once zuul_cloner has new features | 17:56 |
| sdague | I guess I wonder if there is effectively a flush we could issue with those | 17:56 |
| *** hdd has joined #openstack-infra | 17:56 | |
| tchaypo | Or, start hacking those into zuul_cloner immediately | 17:56 |
| *** thedodd has quit IRC | 17:56 | |
| tchaypo | If that seems feasible | 17:56 |
| fungi | sdague: these days zuul tends to be quicker at rereading its config than jenkins-job-builder takes to update on the jenkins masters | 17:56 |
| fungi | race condition basically | 17:56 |
| *** shakamunyi has quit IRC | 17:56 | |
| sdague | gotcha, there is no 2 phase commit logic there? | 17:57 |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: adding support for the custom tools plugin https://review.openstack.org/134011 | 17:57 |
| *** dprince has joined #openstack-infra | 17:57 | |
| fungi | sdague: no, we could enforce it by requiring the job config to go in first and not approve the zuul config addition until after it's applied on the jenkins masters | 17:57 |
| sdague | fungi: oh, gotcha | 17:57 |
| sdague | maybe we should do such a thing? | 17:58 |
| fungi | sdague: but automating that in any meaningful way would probably be more painful than the manual workflow change | 17:58 |
| *** annegent_ has quit IRC | 17:58 | |
| *** derekh has quit IRC | 17:58 | |
| clarkb | fungi could we have ansible do zuul.o.o before jenkins.o.o? | 17:58 |
| fungi | clarkb: possibly, though jjb updates also have a habit of running really, really long and sometimes timing out too | 17:59 |
| *** vigneshvar_ has joined #openstack-infra | 17:59 | |
| *** annegent_ has joined #openstack-infra | 18:00 | |
| fungi | we _could_ do something like we do with the cgit servers and gerrit, of course. just not sure what the variability in duration might be | 18:00 |
| *** amcrn has joined #openstack-infra | 18:00 | |
| *** rmcall has quit IRC | 18:00 | |
| *** dangers is now known as dangers_away | 18:01 | |
| fungi | but as far as actually parsing the change out to figure out what jobs it's adding and then delay zuul config reloading until all added jobs are registered in gearman is probably intractible | 18:01 |
| *** rmcall has joined #openstack-infra | 18:01 | |
| *** shakamunyi has joined #openstack-infra | 18:01 | |
| fungi | or at least likely to be buggy in itself | 18:01 |
| *** gondoi is now known as zz_gondoi | 18:02 | |
| *** gokrokve has quit IRC | 18:03 | |
| *** gokrokve has joined #openstack-infra | 18:03 | |
| *** thedodd has joined #openstack-infra | 18:05 | |
| *** melwitt has joined #openstack-infra | 18:05 | |
| *** luqas has quit IRC | 18:05 | |
| fungi | clarkb: also, shouldn't you be germanying? | 18:07 |
| *** jp_at_hp has quit IRC | 18:07 | |
| clarkb | I should be but larissa is napping | 18:08 |
| clarkb | I have had a liter of kolsch today though | 18:08 |
| arosen1 | fungi: do you see where i'm going wrong with this? http://paste.openstack.org/show/132455/ | 18:08 |
| greghaynes | clarkb: O/ | 18:08 |
| *** SumitNaiksatam has quit IRC | 18:08 | |
| fungi | clarkb: that's puts you two liters down with only a few hours left in the day | 18:09 |
| fungi | better hurry | 18:09 |
| clarkb | greghaynes \o | 18:10 |
| *** mfink__ has quit IRC | 18:10 | |
| *** mfink has joined #openstack-infra | 18:10 | |
| fungi | arosen1: hard to say without seeing the manifest you're using there. is it possible you don't have the nodepool puppet module installed? | 18:10 |
| *** doug-fish has left #openstack-infra | 18:11 | |
| *** mfink has quit IRC | 18:11 | |
| fungi | arosen1: oh, i see, no outer manifest. you're supplying it on the command line | 18:11 |
| *** mfink has joined #openstack-infra | 18:11 | |
| *** hdd has quit IRC | 18:12 | |
| arosen1 | fungi: I'm just trying to install each puppet module individually. Is this not the right way to go about it? | 18:12 |
| *** andreykurilin_ has joined #openstack-infra | 18:13 | |
| fungi | arosen1: i think you may want ::nodepool there | 18:14 |
| fungi | arosen1: here's the manifest we're using to apply it on nodepool.openstack.org... http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/nodepool_prod.pp#n36 | 18:15 |
| *** hdd has joined #openstack-infra | 18:15 | |
| arosen1 | fungi: ah okay I'll start with that. | 18:15 |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: add support to git for changelog against branch https://review.openstack.org/134014 | 18:16 |
| *** Ryan_Lane has joined #openstack-infra | 18:18 | |
| *** weshay has quit IRC | 18:19 | |
| *** weshay has joined #openstack-infra | 18:19 | |
| *** shashankhegde has joined #openstack-infra | 18:21 | |
| *** alexpilotti has joined #openstack-infra | 18:21 | |
| *** atiwari has joined #openstack-infra | 18:22 | |
| *** amcrn has quit IRC | 18:23 | |
| arosen1 | fungi: cool i'm able to run this. Should I do : git clone http://git.openstack.org/cgit/openstack-infra/system-config /etc/project-config first before i run this? Or is something else supposed to do that for me? | 18:23 |
| arosen1 | when i run this I get this failure: | 18:23 |
| arosen1 | Error: Execution of '/usr/bin/git clone /etc/project-config' returned 128: fatal: repository '' does not exist | 18:23 |
| *** bdpayne has joined #openstack-infra | 18:24 | |
| *** thedodd has quit IRC | 18:24 | |
| fungi | arosen1: that's very odd. wonder where it got /etc/project-config as the clone url | 18:24 |
| dhellmann | fungi, clarkb : what is the process for moving a repository to the attic? for example, oslo.version | 18:25 |
| fungi | dhellmann: it's a project rename | 18:25 |
| arosen1 | fungi: i had run: sudo puppet apply --verbose -e "class {'openstack_project::nodepool_prod': mysql_password => 'password', mysql_root_password => 'password'}" | 18:25 |
| *** thedodd has joined #openstack-infra | 18:25 | |
| fungi | dhellmann: needs a project-config change and then link it in the infra meeting agenda | 18:25 |
| dhellmann | fungi: I can rename things in project-config? | 18:26 |
| *** armax has joined #openstack-infra | 18:26 | |
| fungi | dhellmann: so we can discuss scheduling the gerrit outage | 18:26 |
| fungi | dhellmann: yep | 18:26 |
| dhellmann | yeah, this is really really low priority | 18:26 |
| * dhellmann goes to look at how renames work in project-config | 18:26 | |
| fungi | dhellmann: right, for retiring projects we usually batch those up with stackforge project renames and then pull the trigger on them all when a more important rename comes along which has some urgency | 18:27 |
| *** ildikov has joined #openstack-infra | 18:27 | |
| dhellmann | fungi: cool, I just want to make sure I'm on that list at some point | 18:27 |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: add support for pyton and tox matrix project axis https://review.openstack.org/134025 | 18:27 |
| fungi | dhellmann: basically you'd change the project name to openstack-attic/oslo.version in gerrit/projects.yaml, change its acl to look like the acls of other projects in the attic (basically read-only for new changes) and then remove all other sections of files in project-config referencing oslo.version | 18:28 |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: remove the dsvm-cells job from libraries https://review.openstack.org/134028 | 18:29 |
| sdague | dhellmann: so... I think the cells job was the reason oslo was still coupled | 18:29 |
| dhellmann | fungi: ok, thanks | 18:29 |
| dhellmann | sdague: do we want that? probably? | 18:30 |
| fungi | dhellmann: and that change would generally be the final step after you've already made sure no supported branches of any projects use oslo.version, it's no longer referenced in places like devstack/devstack-gate, governance, requirements, et cetera | 18:30 |
| dhellmann | well, I guess only if that job is installing from source | 18:30 |
| dhellmann | fungi: as far as we can tell, it was never released because those features went into pbr instead | 18:30 |
| * dims nods in agreement | 18:31 | |
| arosen1 | fungi: hrm when i run the same puppet apply command it tells me: Error: Could not initialize global default settings: Certificate names must be lower case; see #1168 | 18:31 |
| *** SumitNaiksatam has joined #openstack-infra | 18:31 | |
| fungi | arosen1: does it mention a manifest name, line number, anything corresponding to that error? | 18:32 |
| *** mwagner_lap has joined #openstack-infra | 18:34 | |
| sdague | dhellmann: I don't think we do. We've been running those jobs for 6 weeks not testing anything you think they are testing. | 18:34 |
| sdague | and in reality, they shouldn't actually care about cells or not. If there is a code path that is only triggered in cells, we should figure that out and make it an explicit lib test, not shrug our shoulders and couple the world on the off chance there might be | 18:35 |
| zaro | morning | 18:35 |
| sdague | zaro: hey... so the query you gave me for gerrit core review groups isn't quite right. There is actually a documented syntax for doing the right thing, but it doesn't work | 18:36 |
| *** subscope has joined #openstack-infra | 18:36 | |
| sdague | zaro: label:Code-Review=+1,group=ldap/linux.workflow ... except it seems to completely ignore the contents of group in our env | 18:37 |
| *** thedodd has quit IRC | 18:37 | |
| *** thedodd has joined #openstack-infra | 18:41 | |
| arosen1 | fungi: i think the reason is because the host has a capital letter in its hostname. I think that's what is cause the issue (at least from something I found on google). | 18:42 |
| fungi | arosen1: ahh | 18:43 |
| *** yfried_ has joined #openstack-infra | 18:43 | |
| arosen1 | fungi: is there a script that you guys use to boot strap everything? Do you run: sudo puppet apply --verbose -e "class {'openstack_project::nodepool_prod': mysql_password => 'password', mysql_root_password => 'password'}" manually when you setup your nodepool nodes? | 18:44 |
| zaro | sdague: no workaround for that? | 18:44 |
| zaro | sdague: i don't think i have time to look into that gerrit UI issue this week, maybe next week? | 18:45 |
| fungi | arosen1: we use puppet agent --test with a site.pp of http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp and have it figure out what to apply based on the hostname | 18:45 |
| krotscheck | Debugging this migration would be easier if launchpad wasn’t so abysmally slow | 18:45 |
| zaro | fungi: gah! i created an bad release. help! | 18:45 |
| fungi | zaro: what did you release? | 18:46 |
| zaro | fungi: http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder | 18:46 |
| fungi | zaro: best way to solve it, in general, is to follow up with a good release | 18:46 |
| zaro | fungi: gave wrong release number | 18:46 |
| sdague | zaro: so... interestingly, on gerrit's gerrit the css class for each element is the full java class name that generates it | 18:46 |
| zaro | fungi: 0.4.2 should be 1.0.0 | 18:47 |
| sdague | that almost seems like it's a gwt compile flag instead of gerrit code itself | 18:47 |
| *** achanda has joined #openstack-infra | 18:47 | |
| dhellmann | sdague: I'm fine with decoupling, as long as no one yells at me when we do find those unexpected cases that were only tested by the jobs we're dropping. I'd rather not have to wait for all of those long jobs to run anyway. :-) | 18:47 |
| zaro | fungi: actually should be 0.10.0 | 18:47 |
| *** Guest58449 has joined #openstack-infra | 18:47 | |
| sdague | dhellmann: yeh, you still end up waiting for some long jobs, but this will be a few less | 18:47 |
| fungi | zaro: i think that's probably okay to leave where it is and just add a 0.10.0 tag. we can delete the 0.4.2 tag and tarballs later if they cause any confusion | 18:48 |
| sdague | dhellmann: yeh, well, I'll help fight back about it. It's about being smart about where we test things and what behavior we expect | 18:48 |
| zaro | sdague: fixed upstream? | 18:48 |
| zaro | sdague: maybe we can backport if you can find the fix? | 18:49 |
| dhellmann | sdague: I'm totally on board with being smart about things. Less so with generating anger in the mean time. | 18:49 |
| zaro | fungi: that's great! i'll do that. | 18:49 |
| sdague | zaro: yeh, I don't know. I think someone that understands how to build gerrit to know | 18:50 |
| sdague | me staring at a ton of unfamiliar java code will probably not get results | 18:50 |
| zaro | sdague: ok. i know the maintainer so i'll ask him today. | 18:51 |
| *** arborism has joined #openstack-infra | 18:52 | |
| *** dannywilson has joined #openstack-infra | 18:52 | |
| *** arborism is now known as amcrn | 18:53 | |
| *** e0ne has joined #openstack-infra | 18:54 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/gear: Switched documentation to point to storyboard. https://review.openstack.org/133843 | 18:54 |
| zaro | sdague: is there a workaround for the query you wanted? | 18:55 |
| *** dannywilson has quit IRC | 18:56 | |
| fungi | okay, jenkins05 is cleaned up and restarted, has new workers and is running jobs again. much snappier now | 18:56 |
| arosen1 | fungi: I guess you also have a hiera.yaml file that has a line in it something like this? | 18:57 |
| arosen1 | openstack_project::nodepool_prod: nodepool_mysql_password password | 18:57 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/gearman-plugin: Switched documentation to point to storyboard. https://review.openstack.org/133844 | 18:57 |
| sdague | zaro: no, so the query you gate (reviewerin:nova-core AND ...) only means that someone in nova core reviewed, not that their vote is the vote in question | 18:57 |
| fungi | arosen1: yeah, so that we can keep our passwords secret | 18:57 |
| sdague | and the lable....,group= looks like it should be right.... | 18:57 |
| sdague | however | 18:57 |
| sdague | it returns results as if group isn't checked at all | 18:58 |
| fungi | krotscheck: were you able to figure out what was missing in the migration script causing that duplicate key error? | 18:59 |
| krotscheck | fungi: Well, the duplicate key error was as a result of a previous failed migration | 18:59 |
| *** sabeen3 has joined #openstack-infra | 19:00 | |
| krotscheck | I solved that, I’m just trying to figure out WHY it failed in the first place. | 19:00 |
| fungi | krotscheck: okay, got it | 19:00 |
| *** sabeen has quit IRC | 19:00 | |
| sweston | krtaylor: pleia2: ready? | 19:01 |
| *** armax has quit IRC | 19:01 | |
| krotscheck | fungi: Let me toss up the duplicate key patch and maybe we can get useful output from the live migration, I’m getting no love from my localhost. | 19:01 |
| fungi | krotscheck: fine by me | 19:01 |
| *** gokrokve has quit IRC | 19:02 | |
| *** gokrokve has joined #openstack-infra | 19:02 | |
| pleia2 | sweston: hey | 19:03 |
| sweston | pleia2: hello | 19:03 |
| *** otherwiseguy has quit IRC | 19:04 | |
| *** pelix has quit IRC | 19:04 | |
| sweston | pleia2: waiting on Kurt ... would you like to go over the etherpad? https://etherpad.openstack.org/p/Third-Party-CI-Dashboard-InitialPlanning | 19:05 |
| pleia2 | sweston: so hopefully krtaylor will wander in, I was just browsing https://etherpad.openstack.org/p/Third-Party-CI-Dashboard-InitialPlanning to figure out some next steps | 19:05 |
| pleia2 | :D | 19:05 |
| *** talluri has quit IRC | 19:05 | |
| sweston | pleia2: hehe, ok | 19:05 |
| *** radez_g0` is now known as radez | 19:06 | |
| *** marun has quit IRC | 19:06 | |
| *** tsg_ has joined #openstack-infra | 19:06 | |
| *** weshay has quit IRC | 19:07 | |
| sweston | pleia2: so, then we could use the last known good tree as calibration? Does this mean last known good tree of the repository, or last known good run of the CI system against the repository? | 19:07 |
| *** gokrokve has quit IRC | 19:07 | |
| pleia2 | sweston: good question, I'm not sure | 19:07 |
| *** weshay has joined #openstack-infra | 19:08 | |
| sweston | pleia2: ok | 19:08 |
| *** koolhead17 has quit IRC | 19:08 | |
| pleia2 | looks like inclination from the pad is known good tree | 19:08 |
| pleia2 | maybe we ask for some nova folks to chime in too | 19:09 |
| *** koolhead17 has joined #openstack-infra | 19:09 | |
| *** alexpilotti_ has joined #openstack-infra | 19:09 | |
| *** liusheng has quit IRC | 19:09 | |
| *** alexpilotti has quit IRC | 19:09 | |
| *** alexpilotti_ is now known as alexpilotti | 19:09 | |
| *** liusheng has joined #openstack-infra | 19:09 | |
| *** gokrokve has joined #openstack-infra | 19:10 | |
| *** tsg_ has quit IRC | 19:10 | |
| *** rmcall has quit IRC | 19:10 | |
| *** bswartz has quit IRC | 19:11 | |
| *** ZZelle_ has joined #openstack-infra | 19:11 | |
| sweston | pleia2: so, then what is last known good tree? master? what if that particular system failed on that run as well? | 19:11 |
| *** rmcall has joined #openstack-infra | 19:12 | |
| pleia2 | sweston: hm, right, that is a possibility if reviewers are merging code that was -1ed by a system | 19:12 |
| *** shakamunyi has quit IRC | 19:12 | |
| krtaylor | sweston, pleia2 here now | 19:12 |
| * krtaylor reading scrollback | 19:12 | |
| *** shakamunyi has joined #openstack-infra | 19:13 | |
| sweston | krtaylor: great! please jump in | 19:13 |
| *** koolhead17 has quit IRC | 19:13 | |
| krtaylor | sweston, pleia2 - the etherpad looks good, I think there is plenty of content for a spec | 19:15 |
| pleia2 | I agree, there are some questions but we can work those out in the spec | 19:16 |
| krtaylor | sweston, pleia2 - it looks like the only real question is the process for determining the failure state | 19:16 |
| pleia2 | yeah | 19:16 |
| krtaylor | but there are several good options listed | 19:16 |
| sweston | krtaylor: so, my current question is this, what are we defining as the last known good tree? | 19:16 |
| sweston | krtaylor: yes, that has always been the sticky issue | 19:17 |
| sdague | clarkb: do you know the tempest patch that mtreinish had up for the secgroup race? | 19:17 |
| clarkb | sdague ya one sec | 19:17 |
| krtaylor | sweston, good question - jhesketh and I discussed just finding the last good run, meaning success, but we can build to that | 19:18 |
| krtaylor | baby steps | 19:18 |
| clarkb | sdague https://review.openstack.org/#/c/132729/ | 19:18 |
| sweston | krtaylor: ok | 19:19 |
| sweston | krtaylor: next baby step? | 19:19 |
| sweston | pleia2: ^ | 19:19 |
| pleia2 | I'd say taking the first stab at a spec | 19:20 |
| krtaylor | sweston, so, it also kinda goes back to the conversation with jhesketh | 19:20 |
| krtaylor | yes, spec | 19:20 |
| krtaylor | then cmd line tool, similar to jhesketh 's | 19:20 |
| krtaylor | then webui | 19:20 |
| krtaylor | then filtering | 19:20 |
| krtaylor | then rerun logic | 19:21 |
| *** gokrokve has quit IRC | 19:21 | |
| krtaylor | there is probably a few more steps in there | 19:21 |
| *** gokrokve has joined #openstack-infra | 19:21 | |
| *** gokrokve has quit IRC | 19:22 | |
| tchaypo | and then I discover https://gemfury.com/l/pypi-server and sheep quietly in the corner | 19:22 |
| sweston | krtaylor: pleia2: so, in order to make this as modular as possible, I'd like to put the logic into the rest api, so that the command line tool and the webui can call the api | 19:22 |
| sweston | ^ sound ok? | 19:22 |
| *** gokrokve has joined #openstack-infra | 19:22 | |
| sweston | additionally, that will enable data collection to run as a server process, instead of messy cron jobs or scheduling | 19:23 |
| krtaylor | sweston, I *think* so, I guess I was thinking start smaller, but, I don't know where you are currently | 19:23 |
| pleia2 | I don't have a strong opinion either way | 19:24 |
| arosen1 | fungi: why do you use puppet agent rather then puppet apply? | 19:24 |
| *** _shaps_ has quit IRC | 19:24 | |
| sweston | it's much easier this way, rather than going back and re-factoring later | 19:24 |
| fungi | arosen1: because we're still using a puppet master at the moment | 19:24 |
| sweston | I can put all of that in the spec | 19:25 |
| pleia2 | sweston: sounds good | 19:25 |
| krtaylor | sweston, sound like a plan, I am not opposed to that, I was starting to play with ideas too | 19:25 |
| krtaylor | sweston, do you have a personal repo started? | 19:25 |
| sweston | krtaylor: pleia2: ok great. if there are any more ideas, feel free to send an email, or irc, or put them in the etherpad | 19:25 |
| fungi | tchaypo: beware --extra-index-url has some implied risk... if you have a package name which is not registered on pypi but is present in your private index, someone can come along later and upload something to pypi with the same name and you'll silently start installing that instead | 19:26 |
| pleia2 | thanks sweston and krtaylor | 19:26 |
| sweston | krtaylor: code on my dev machine right now, but I will have a server up soon where people can check out the code | 19:26 |
| tchaypo | first roadblock to using zuul-cloner - I can’t find it. It doesn’t seem to be installed with “pip install zuul" | 19:26 |
| arosen1 | fungi: sorry for my noob questions. But I guess in my case I'll use puppet apply since I don't have a puppet master setup? | 19:26 |
| tchaypo | fungi: would my sentence read differently if “sheep” was replaced with “weeps”? | 19:26 |
| fungi | arosen1: yep, should work fine | 19:26 |
| *** armax has joined #openstack-infra | 19:26 | |
| *** marun has joined #openstack-infra | 19:27 | |
| fungi | tchaypo: probably not. i'll just picture you weeping over some sheep | 19:27 |
| krtaylor | sweston, sounds good, let me know or put in the etherpad | 19:27 |
| *** dannywilson has joined #openstack-infra | 19:27 | |
| tchaypo | deep makes sense to me, because it gets around that issue - only need one index, and it’s local, and it overrides the upstream namespace | 19:27 |
| *** vigneshvar__ has joined #openstack-infra | 19:27 | |
| arosen1 | fungi: in the infra case you guys have a master and agents so that you can just update the puppet config files on the master and it will push them out to all your slaves? | 19:27 |
| tchaypo | s/deep/devpi/ | 19:27 |
| clarkb | tchaypo you likely need a source install | 19:27 |
| * tchaypo needs to turn off autocorrect in this window | 19:27 | |
| krtaylor | sweston, want to discuss the spec at the next third-party meeting? | 19:27 |
| fungi | arosen1: sort of. we actually orchestrate it with ansible now, so could probably switch to masterless and puppet apply instead (mordred has plans for how this might work) | 19:28 |
| sweston | krtaylor: yes, that would be great. is the agenda modifiable? | 19:28 |
| pleia2 | sweston: yeah, see 11:27:29 < arosen1> fungi: in the infra case you guys have a master and agents so that you can just update the puppet config files on the master and it will push them out to all your slaves? | 19:28 |
| pleia2 | gah | 19:28 |
| pleia2 | https://wiki.openstack.org/wiki/Meetings/ThirdParty | 19:28 |
| pleia2 | pesky paste lag | 19:28 |
| fungi | tchaypo: clarkb: yes, i think it was added since the most recent release, so there's no released version of zuul including it _yet_ | 19:28 |
| pleia2 | sweston: just scroll down & edit 11/17/14 agenda | 19:29 |
| tchaypo | so in other words | 19:29 |
| tchaypo | my tool that has been in use for a long time | 19:29 |
| tchaypo | is being pushed back on because zuul has an unreleased feature that doesn’t do what I need? | 19:29 |
| krtaylor | sweston, yes, feel free to add to the agenda, or I can whatever | 19:29 |
| sweston | pleia2: ok, will do. your previous message is addressing delivery of the code with puppet? I am set up for that | 19:29 |
| pleia2 | sweston: previous message was a mispaste :) | 19:30 |
| sweston | krtaylor: I'll add it, this afternoon | 19:30 |
| fungi | tchaypo: that's how i'd interpret it. note i wasn't pushing back, just suggesting that if it's something tripleo is developing/using then it should be openstack not stackforge | 19:30 |
| tchaypo | (I do think it makes sense to have one tool to do the job, it’s just the timing that makes me cry) | 19:30 |
| sweston | pleia2: ah, ok | 19:30 |
| clarkb | its being pushed back on because I thought you wanted to replace zuul in the gate | 19:30 |
| sweston | krtaylor: pleia2: anything else? | 19:30 |
| clarkb | apparently I was wrong | 19:30 |
| *** achanda has quit IRC | 19:30 | |
| pleia2 | I'm on vacation next week, but there are plenty of folks around for the meeting | 19:30 |
| pleia2 | and I'm 100x better at vacation than clarkb | 19:30 |
| *** mpaolino has quit IRC | 19:30 | |
| pleia2 | ;D | 19:30 |
| clarkb | pleia2 :P | 19:30 |
| fungi | clarkb: i am often wrong on the internet | 19:31 |
| *** vigneshvar_ has quit IRC | 19:31 | |
| sweston | pleia2: 😃 | 19:31 |
| krtaylor | hehheh | 19:31 |
| *** achanda has joined #openstack-infra | 19:31 | |
| tchaypo | heh | 19:31 |
| pleia2 | sweston: I think that's it, thanks for handling the spec | 19:32 |
| clarkb | tchaypo also I thought you guys CD | 19:32 |
| *** ivar-laz_ has joined #openstack-infra | 19:32 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Create repo for oslo.context https://review.openstack.org/132877 | 19:32 |
| sweston | krtaylor: pleia2 ok, thanks for your help today | 19:32 |
| clarkb | releasing features is so 2010 | 19:32 |
| krtaylor | sweston, pleia2 yes, sounds like a plan | 19:32 |
| sweston | pleia2: have a nice vacation 😃 | 19:32 |
| tchaypo | so did I, until i started looking at hp2 | 19:32 |
| tchaypo | i have zulu-clone now but it seems to want 8 zuul_* parameters | 19:33 |
| *** ivar-laz_ has quit IRC | 19:33 | |
| tchaypo | *reads docs* | 19:33 |
| *** achanda has quit IRC | 19:34 | |
| tchaypo | looking at http://ci.openstack.org/zuul/launchers.html?highlight=zuul_ref it looks like ZUUL_CHANGE_IDS already has roughly the information I’d want to provide, but zuul-cloner doesn’t use that variable | 19:35 |
| *** achanda has joined #openstack-infra | 19:35 | |
| *** ivar-lazzaro has quit IRC | 19:35 | |
| *** ivar-laz_ has joined #openstack-infra | 19:35 | |
| clarkb | tchaypo likely not as its redundant for current cloner use. the cloner usesthe zuul ref instead | 19:36 |
| clarkb | its this ref that would either be extended or have another ref list option to use instead | 19:37 |
| *** Guest58449 has quit IRC | 19:37 | |
| pleia2 | sweston: thanks :) | 19:37 |
| tchaypo | from http://ci.openstack.org/zuul/merger.html?highlight=ref it looks like the ref is of the form refs/zuul/<branch>/Z<random sha1> | 19:37 |
| clarkb | tchaypo yup but it could be an arbitrary git ref | 19:38 |
| tchaypo | I think I can use that to check out trunk - I have to find a ref that matches the shah of the tip of trunk | 19:38 |
| clarkb | eg master | 19:38 |
| tchaypo | oh, if it can be an arbitrary ref that’s easy | 19:38 |
| fungi | tchaypo: for that matter like i suggested earlier you could use arbitrary local branches of changes, or lightweight tags with a common name across all repos, or whatever | 19:39 |
| fungi | depending on what's convenient | 19:39 |
| tchaypo | “local branches of changes” - as in, check out trunk, merge in the patches I need, then use that? | 19:40 |
| tchaypo | apologies if I sound antagonistic - I’m trying to wrap my head around zuul to see if I can use it, but it’s a lot to take in at once | 19:41 |
| arosen1 | fungi: when i run: openstack_project::nodepool_prod it fails because g++ isn't installed on the box. Is there another puppet module I should install first which gets this? | 19:43 |
| *** pblaho has quit IRC | 19:43 | |
| arosen1 | the image i'm using is just a min install of 14.04-server | 19:43 |
| fungi | arosen1: it's likely that rackspace's ubuntu 14.04 images come with that preinstalled so we never noticed | 19:44 |
| arosen1 | fungi: should I update add it to nodepool/manifests/init.pp? | 19:46 |
| tchaypo | zuul-cloner --zuul-ref master --zuul-branch master --zuul-change 0 --zuul-patchset 0 --zuul-pipeline none --zuul-project none --zuul-url none git://git.openstack.org openstack-infra/project-config openstack/tripleo-incubator | 19:47 |
| fungi | arosen1: can't hurt. likely we should just add all of build-essential to our servers to be on the safe side | 19:47 |
| tchaypo | seems to do okay at cloning the two repos | 19:47 |
| *** timrc is now known as timrc-afk | 19:47 | |
| openstackgerrit | Aaron Rosen proposed openstack-infra/system-config: g++ is also required to install nodepool https://review.openstack.org/134038 | 19:48 |
| clarkb | fungi arosen1 I think we install build essential everywhere | 19:49 |
| *** MarkAtwood has joined #openstack-infra | 19:49 | |
| *** baoli has quit IRC | 19:50 | |
| arosen1 | gotcha | 19:50 |
| *** baoli has joined #openstack-infra | 19:51 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Clean recovery for import. https://review.openstack.org/134039 | 19:53 |
| *** rlandy has joined #openstack-infra | 19:53 | |
| fungi | clarkb: i thought that was just on slaves... checking | 19:53 |
| *** bdpayne has quit IRC | 19:54 | |
| fungi | clarkb: yeah, etherpad and jenkins slaves, not everywhere | 19:54 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Clean recovery for import. https://review.openstack.org/134039 | 19:55 |
| fungi | krotscheck: so just that ^ ? | 19:55 |
| *** rfolco has quit IRC | 19:55 | |
| krotscheck | fungi: So far, yeah. That patch will just make sure that the script recovers on any failure. | 19:55 |
| krotscheck | and continues where it left off. | 19:55 |
| krotscheck | I still haven’t been able to get the comments dump that you referred to to happen. | 19:56 |
| fungi | krotscheck: one thing i didn't think to check... is it possible someone opened a story on storyboard while the script was running, or how did we end up resolving the "could create conflicting story numbers" dilemma? | 19:56 |
| krotscheck | fungi: Also, the console output will be a bit more specific about what it’s doing in comments. | 19:56 |
| *** achanda has quit IRC | 19:56 | |
| krotscheck | fungi: We set the auto increment to something Large (TM) so new id’s happen there. | 19:57 |
| fungi | krotscheck: oh, perfect. not a risk then | 19:57 |
| krotscheck | fungi: Yep. | 19:57 |
| *** e0ne has quit IRC | 19:57 | |
| *** sabeen1 has joined #openstack-infra | 19:58 | |
| *** sabeen3 has quit IRC | 19:58 | |
| *** jerryz has quit IRC | 20:00 | |
| fungi | krotscheck: okay, i have master storyboard with that patch installed in an active venv. i can retry the import now unless you need me to hold off | 20:01 |
| *** subscope has quit IRC | 20:01 | |
| *** rushiagr is now known as rushiagr_away | 20:01 | |
| *** beekneemech has joined #openstack-infra | 20:02 | |
| *** bnemec has quit IRC | 20:04 | |
| arosen1 | fungi: Is the syntax like this that you have in /etc/puppet/hiera.yaml ; openstack_project::nodepool_prod: nodepool_mysql_password xxxx | 20:04 |
| arosen1 | I tried a few things: Openstack_project::Nodepool_prod: mysql_root_password password | 20:04 |
| arosen1 | puppet still is complaining though about: Error: Must pass mysql_root_password to Class[Openstack_project::Nodepool_prod] at line 1 on node puppet2 | 20:04 |
| *** liam_ has joined #openstack-infra | 20:05 | |
| fungi | arosen1: just 'nodepool_mysql_password: xxxx' in the hiera file (it's not scoped) | 20:05 |
| *** liam_ is now known as Guest39243 | 20:05 | |
| krotscheck | fungi: Go ahead. | 20:05 |
| *** Guest39243 has quit IRC | 20:06 | |
| krotscheck | fungi: Pipe the output if you don’t mind. | 20:06 |
| *** annegent_ has quit IRC | 20:07 | |
| arosen1 | fungi: http://paste.openstack.org/show/132511/ :/ | 20:07 |
| *** annegent_ has joined #openstack-infra | 20:08 | |
| *** aysyd has quit IRC | 20:08 | |
| *** marun has quit IRC | 20:09 | |
| fungi | arosen1: oh, you're operating at a level below where we apply those from hiera. we map that in our site.pp | 20:09 |
| fungi | arosen1: the Openstack_project::Nodepool_prod class is just expecting those to be passed in as normal puppet variables | 20:11 |
| dims | AJaeger: so looking closer at what i have, i don't need translation jobs | 20:11 |
| dims | AJaeger: fixing the review | 20:11 |
| fungi | arosen1: er, as normal class parameters | 20:11 |
| *** ChuckC has quit IRC | 20:11 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Create repo for oslo.context https://review.openstack.org/132877 | 20:12 |
| dims | AJaeger: thanks for spotting that | 20:12 |
| *** doug-fish has joined #openstack-infra | 20:12 | |
| nibalizer | arosen1: if you want to you can do it that way | 20:13 |
| nibalizer | you would set the key openstack_project::nodepool_prod::nodepool_mysql_password: xxxxx | 20:13 |
| nibalizer | this is a thing called data bindings | 20:14 |
| nibalizer | whose purpose is brevity and confusion | 20:14 |
| *** timrc-afk is now known as timrc | 20:15 | |
| *** jcoufal has quit IRC | 20:15 | |
| sdague | clarkb / AJaeger - how you guys feeling about this? should definitely drop gate queue depth | 20:17 |
| *** andreykurilin__ has joined #openstack-infra | 20:17 | |
| *** andreykurilin_ has quit IRC | 20:17 | |
| arosen1 | fungi: can I see a snip of how you guys are doing it? I'm getting the same error with openstack_project::nodepool_prod::mysql_root_password: password | 20:18 |
| *** marun has joined #openstack-infra | 20:18 | |
| fungi | arosen1: like i tried to explain, openstack_project::nodepool_prod isn't checking hiera | 20:18 |
| arosen1 | ah i see sorry. | 20:19 |
| fungi | arosen1: this is where we tie it in... http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n450 | 20:19 |
| *** AlexF has joined #openstack-infra | 20:19 | |
| *** mjturek has quit IRC | 20:20 | |
| fungi | arosen1: we're having site.pp pull those variables from hiera and then pass them into openstack_project::nodepool_prod as class params | 20:20 |
| *** otherwiseguy has joined #openstack-infra | 20:20 | |
| fungi | arosen1: then from there they trickle down into the nodepool module classes | 20:20 |
| *** prad has joined #openstack-infra | 20:22 | |
| fungi | krotscheck: unfortunately when redirecting output from storyboard-migrate its stdout/stderr end up buffered, so i can't really tell what it's doing yet, but at least it hasn't terminated | 20:22 |
| arosen1 | fungi: i see, I guess there is no way to run the class openstack_project::nodepool_prod which checks hiera() since it's scoped with node 'nodepool.openstack.org'? | 20:22 |
| krotscheck | fungi: Works. | 20:22 |
| krotscheck | fungi: No news is good news :) | 20:22 |
| arosen1 | fungi: or would you guys be willing to accept a change which adds hiera() to the config options in class nodepool() so one doesn't have to depend on the hostname? | 20:23 |
| fungi | arosen1: well, you could copy the block you see there in site.pp and modify it to suit your needs, or even just embed the literal values you want rather than using hiera. we mainly do the hiera dance so that we can avoid publishing password values | 20:24 |
| arosen1 | fungi: I don't want to deviate the modules really away from what you guys have. I guess I'll just pass these things in as vars for now | 20:25 |
| *** otherwiseguy has quit IRC | 20:25 | |
| fungi | arosen1: passing them in as vars is what we're doing there | 20:25 |
| asselin | arosen1, for my scripts, it get's passed in directly and used here: https://github.com/rasselin/os-ext-testing/blob/master/puppet/modules/os_ext_testing/manifests/master.pp#L334 | 20:25 |
| fungi | arosen1: if i understand what you're suggesting, there will always be some level of manifests in our puppet which you'll need to diverge on | 20:26 |
| *** andreykurilin_ has joined #openstack-infra | 20:26 | |
| fungi | arosen1: we can't move every last bit of our configuration into hiera and wouldn't want to anyway | 20:26 |
| arosen1 | fungi: i know but i don't want to set my hostnames to the ones you have defined in site.pp i.e nodepool.openstack.org | 20:26 |
| *** andreykurilin__ has quit IRC | 20:26 | |
| fungi | arosen1: i'm not suggesting you reuse our site.pp | 20:26 |
| fungi | arosen1: i'm suggesting you use something similar to our site.pp, modified for your use case | 20:27 |
| arosen1 | ah okay :) i'll try that. | 20:27 |
| arosen1 | sorry about the confusion. | 20:27 |
| fungi | arosen1: if you haven't seen it, we discuss options like that in http://ci.openstack.org/sysadmin.html#making-a-change-in-puppet | 20:27 |
| *** mmaglana has joined #openstack-infra | 20:28 | |
| fungi | arosen1: what i'm suggesting is basically the local.pp workflow shown there, but of course there are all sorts of ways to skin it | 20:28 |
| *** mpaolino has joined #openstack-infra | 20:28 | |
| *** mpaolino has quit IRC | 20:28 | |
| *** cpowell has joined #openstack-infra | 20:29 | |
| nibalizer | fungi: arosen1 one of the ways we could make this easier is by changing our site.pp to node /nodepool/ { } | 20:29 |
| *** marun has quit IRC | 20:30 | |
| cpowell | Greetings, I am working on understanding the openstack CI process and I am having alittle trouble locating a script in a Jenkins job. Could someone point me to where I can find https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/magnetodb.yaml#L52 | 20:30 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Create repo for oslo.context https://review.openstack.org/132877 | 20:30 |
| *** bswartz has joined #openstack-infra | 20:30 | |
| waynr | zaro: i'm just about finished with the plugins_info changes on JJB but i have a question for ya | 20:31 |
| *** krtaylor has quit IRC | 20:31 | |
| waynr | to test the test subcommand i am using the hipchat notification plugin module change that i am developing in parallel | 20:32 |
| dims | cpowell: http://git.openstack.org/cgit/stackforge/magnetodb/tree/tools/install_cassandra_ccm.sh | 20:32 |
| krotscheck | fungi: If it makes you feel any better, the import script I kicked off this morning is still running. | 20:32 |
| waynr | so the two gerrit reviews i have open now will become one that inclues both the plugin_info changes and the hipchat notification changes | 20:32 |
| cpowell | dims: Thank you! | 20:32 |
| *** ivar-laz_ has quit IRC | 20:33 | |
| *** punith has quit IRC | 20:33 | |
| *** alexpilotti has quit IRC | 20:33 | |
| waynr | but it seems almost like a bad idea to me to complect the unit test for the 'test' subcommand with hipchat notifier specific behavior | 20:33 |
| waynr | if you have any advice for how to better test the plugins_info behavior under the 'test' subcommand, please let me know | 20:34 |
| *** mmaglana has quit IRC | 20:34 | |
| *** mmaglana_ has joined #openstack-infra | 20:34 | |
| *** Sincler has quit IRC | 20:34 | |
| *** mfink has quit IRC | 20:36 | |
| *** dizquierdo has joined #openstack-infra | 20:36 | |
| *** ChuckC has joined #openstack-infra | 20:38 | |
| dstufft | dhellmann: nice, dropping the namespaces | 20:38 |
| fungi | dstufft: if only you could have been there for the summit session! | 20:38 |
| ekarlso | Hi, how can I find out who can approve patches for the cue project ? | 20:39 |
| *** vigneshvar__ has quit IRC | 20:39 | |
| *** achanda has joined #openstack-infra | 20:39 | |
| *** achanda has quit IRC | 20:39 | |
| fungi | ekarlso: i think you should be able to tell by looking at its gerrit acl | 20:39 |
| dstufft | fungi: heh, yea :/ | 20:39 |
| *** _nadya_ has joined #openstack-infra | 20:39 | |
| dstufft | is there a location for the next one? | 20:39 |
| fungi | dstufft: vancouver, bc | 20:40 |
| fungi | dstufft: mid-may timeframe if memory serves | 20:40 |
| ekarlso | who creates the cue-core then ? | 20:40 |
| ekarlso | or adds members to it :p | 20:41 |
| fungi | ekarlso: the group is automatically created when the acl is applied, but it starts out empty and then the proposer of the addition needs to let an infra gerrit admin know who to put in as the initial member | 20:41 |
| ekarlso | @ vipul ^ | 20:41 |
| krotscheck | dstufft: https://www.openstack.org/summit/vancouver-2015/ | 20:42 |
| *** bdpayne has joined #openstack-infra | 20:42 | |
| fungi | krotscheck: glad to see working for them hasn't dulled my memory | 20:42 |
| *** krtaylor has joined #openstack-infra | 20:42 | |
| *** achanda has joined #openstack-infra | 20:43 | |
| dhellmann | dstufft: yeah, I wish we could make that work but I think we're all tired of fighting it at this point | 20:43 |
| dstufft | it's really a failing of the tooling :/ | 20:43 |
| *** _nadya_ has quit IRC | 20:43 | |
| dstufft | but I think it's the best option | 20:44 |
| fungi | krotscheck: same error as i was getting before that change. here's what i got redirecting stdout and stderr... http://fungi.yuggoth.org/tmp/storyboard-migrate/ | 20:44 |
| dhellmann | dstufft: yeah, I hear its a lot better in 3.x, but we tabled any discussion of moving back to using them | 20:44 |
| dstufft | fungi: krotscheck ah neat, will have to try to make it | 20:44 |
| krotscheck | fungi: Oh! | 20:44 |
| krotscheck | Data too long for column 'content' at row 1" | 20:44 |
| * krotscheck checks things. | 20:44 | |
| dstufft | dhellmann: yea.. 3.x adoption is sad | 20:44 |
| fungi | krotscheck: yep, now that i have the start of the output, it's a bit more obvious | 20:45 |
| krotscheck | fungi: Wow, content is a text column. | 20:45 |
| dstufft | dhellmann: % of what version pip / setuptools claims it's running under when downloading from PyPI - http://d.stufft.io/image/1c360Y3J220D ( old data, I'm updating it and will probably do a blog or something when I update ) | 20:46 |
| krotscheck | 230K characters in that comment. | 20:47 |
| fungi | krotscheck: yep... that's <2^18 | 20:47 |
| fungi | are mysql text columns limited to 2^16? | 20:47 |
| *** ivar-lazzaro has joined #openstack-infra | 20:47 | |
| krotscheck | fungi: Looks like it’s byte based. TEXT is 65K, MEDIUMTEXT is 16M, LongText is 4G | 20:48 |
| krotscheck | Lemme see if that’s something I can modify with alembic/sqla | 20:48 |
| krotscheck | But even so: Holy smokes | 20:49 |
| fungi | krotscheck: okay, yep, so 2^16 for text column type | 20:49 |
| fungi | which that exceeds | 20:49 |
| *** shakamunyi_ has joined #openstack-infra | 20:49 | |
| fungi | mediumtext being 2^24 and longtext 2^32 apparently | 20:49 |
| vipul | fungi: can you add me to cue-core? it just got created | 20:49 |
| fungi | vipul: | 20:50 |
| fungi | er | 20:50 |
| fungi | vipul: done | 20:50 |
| dhellmann | dstufft: that's only a little surprising to me, actually. Lots more libs are supporting multiple versions, but app support is lagging. | 20:50 |
| *** e0ne has joined #openstack-infra | 20:50 | |
| *** bcourt has quit IRC | 20:51 | |
| *** signed8bit has quit IRC | 20:51 | |
| krotscheck | Bah, 64K should be enough for anyone. | 20:51 |
| fungi | vipul: also added you to cue-release | 20:51 |
| vipul | aweome thanks fungi | 20:51 |
| fungi | yw | 20:51 |
| *** ivar-lazzaro has quit IRC | 20:52 | |
| *** ivar-lazzaro has joined #openstack-infra | 20:53 | |
| sdague | fungi: I'm wondering for new projects if we shouldn't default to not having a -release group, and just a -core group instead | 20:53 |
| *** ivar-lazzaro has joined #openstack-infra | 20:54 | |
| *** hashar has joined #openstack-infra | 20:54 | |
| sdague | and have projects only get a separate release group if they ask specifically | 20:54 |
| sdague | because I think for smaller groups, having all of core be able to do releases is probably more highly available | 20:54 |
| *** mjturek has joined #openstack-infra | 20:55 | |
| *** weshay has quit IRC | 20:56 | |
| *** dprince has quit IRC | 20:56 | |
| *** mjturek has quit IRC | 20:56 | |
| fungi | sdague: maybe. the converse issue though is that as soon as the core group grows past about 2 people they're not all necessarily as well versed in the release process nuances and then more often create disasters they want us to help them clean up | 20:57 |
| *** gyee has quit IRC | 20:58 | |
| fungi | sdague: so it was seen as a sort of future-proofing | 20:58 |
| *** signed8bit has joined #openstack-infra | 20:58 | |
| *** AlexF has quit IRC | 20:59 | |
| arosen1 | fungi: okay I've adapted your site.pp to match my hostnames: though the hiera.yaml doesn't seem to right as it's using the default XXX password: | 20:59 |
| arosen1 | openstack_project::nodepool_prod::nodepool_mysql_root_password: password | 21:00 |
| arosen1 | seems to match this example with ntp :/ https://docs.puppetlabs.com/hiera/1/complete_example.html | 21:00 |
| *** tjones has joined #openstack-infra | 21:01 | |
| fungi | arosen1: our hiera just uses 'nodepool_mysql_root_password: password' (without the additional scope | 21:02 |
| fungi | arosen1: though i have no idea if puppet apply will reference hiera for you automatically | 21:02 |
| fungi | nibalizer: ^ do you know (if still around)? | 21:02 |
| *** shakamunyi_ has quit IRC | 21:03 | |
| tjones | Hi we are running into an issue with our internal tempest where test_resize_server_from_manual_to_auto fails in the 1st run and pass in the 2nd. i was wondering if you had seen anything like that also (this is icehouse btw) | 21:04 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Updated size of comment content. https://review.openstack.org/134051 | 21:04 |
| *** adalbas has quit IRC | 21:04 | |
| sdague | fungi: maybe we fix that part? because if more people knew how to do releases it would be tribal knowledge that everyone has, and everyone could help with | 21:05 |
| openstackgerrit | James Carey proposed openstack-dev/hacking: WIP: Add incubator hacking checks https://review.openstack.org/134052 | 21:05 |
| fungi | tjones: no idea... you might ask in #openstack-qa | 21:05 |
| hashar | good evening! | 21:06 |
| zaro | hashar: hi | 21:06 |
| fungi | sdague: well, we do have it documented... but the problem is generally that people don't follow the documentation | 21:06 |
| hashar | zaro: I have yet to fill the bug about the Gearman plugin deadlock slaves executor. But that will come :] | 21:06 |
| fungi | sdague: maybe it's not visible/discoverable enoughb | 21:06 |
| zaro | hashar: i just updated bug 1177831 | 21:07 |
| uvirtbot | Launchpad bug 1177831 in python-jenkins "build_job with parameters fails since POST is not used." [Undecided,In progress] https://launchpad.net/bugs/1177831 | 21:07 |
| tjones | fungi: thanks i'll try there | 21:07 |
| hashar | zaro: and I fixed a python-jenkins test that wasn't passing on py34 ( https://review.openstack.org/#/c/132433/ ) | 21:07 |
| fungi | sdague: it's likely something we'll be putting in the developer-targeted part of infra-manual | 21:08 |
| fungi | so perhaps that will help | 21:08 |
| sdague | fungi: cool | 21:08 |
| *** weshay has joined #openstack-infra | 21:08 | |
| *** marun has joined #openstack-infra | 21:09 | |
| zaro | hashar: does my comment on that bug make sense? | 21:10 |
| krotscheck | fungi: https://review.openstack.org/#/c/134051/ | 21:10 |
| fungi | krotscheck: yep, was just looking | 21:10 |
| *** shakamun_ has joined #openstack-infra | 21:10 | |
| *** shakamunyi has quit IRC | 21:10 | |
| hashar | zaro: I need to attempt to reproduce it against my local setup | 21:10 |
| hashar | zaro: I am wondering whether it could be caused by the option to prevent XSS attacks (adding a crumb ) | 21:11 |
| fungi | krotscheck: though i guess we'll need that actually applied on the server before trying again, since it's not just a fix to the import script this time | 21:11 |
| krotscheck | fungi: Yep. | 21:11 |
| krotscheck | So we’ll need NikitaKonovalov or ttx to take a look. | 21:11 |
| fungi | krotscheck: agreed. the only concern i have with it is the risk it might degrade performance (not sure if there are performance implications of text over mediumtext/longtext) | 21:13 |
| fungi | mordred may also have input there | 21:13 |
| mordred | aroo | 21:14 |
| krotscheck | fungi: Well, if anything there’s transmission performance degradation for just pushing all those bits. | 21:14 |
| krotscheck | mordred: Launchpad has comments that are bigger than 65K bytes. | 21:14 |
| mordred | joy | 21:14 |
| *** gyee has joined #openstack-infra | 21:14 | |
| krotscheck | mordred: so we did this: https://review.openstack.org/134051 | 21:14 |
| mordred | taking off. back in 10 | 21:14 |
| zaro | hashar: i'm pretty ignorant about attacks and i haven't figured out the crumb stuff yet :( | 21:14 |
| fungi | more specifically, one of the openstack-ci bugs on lp is in that situation, so there are likely a lot more | 21:14 |
| *** erikmwilson has joined #openstack-infra | 21:15 | |
| *** mrmartin has quit IRC | 21:15 | |
| hashar | zaro: the idea is that someone can trick you in clicking on a URL that would thus use your credentials | 21:15 |
| zaro | hashar: i should try it on an older LTS version to see if it was possible to pass params to non-parameterized jobs in the past | 21:15 |
| hashar | zaro: so if you are logged in Jenkins with admin rights, an attacker can craft an URL that would end up deleting a job or creating an account or whatever | 21:16 |
| *** markmcclain has quit IRC | 21:16 | |
| *** wenlock has quit IRC | 21:16 | |
| *** wenlock has joined #openstack-infra | 21:16 | |
| *** shakamunyi_ has joined #openstack-infra | 21:17 | |
| zaro | hashar: i'm playing around with docker so if i figure that out then should be easy to configure multiple containers with various jenkins versions. | 21:17 |
| hashar | zaro: definitely. And git-review has some integration tests against a gerrit.war :] | 21:17 |
| zaro | hashar: interesting attack, maybe it is a jenkins security fix. | 21:19 |
| *** wenlock has quit IRC | 21:20 | |
| zaro | hashar: tbh, not real interested in extending tests for python-jenkins at this point. | 21:20 |
| *** wenlock has joined #openstack-infra | 21:20 | |
| zaro | hashar: but that is very good to know. | 21:20 |
| fungi | sdague: fyi, this came up in the infra meeting yesterday as waiting for further feedback from you https://review.openstack.org/122615 | 21:20 |
| *** atiwari has quit IRC | 21:21 | |
| *** e0ne has quit IRC | 21:21 | |
| *** AlexF has joined #openstack-infra | 21:21 | |
| mordred | krotscheck: looks fine to me | 21:23 |
| krotscheck | mordred: No performance concerns? | 21:24 |
| *** wenlock has quit IRC | 21:25 | |
| *** wenlock has joined #openstack-infra | 21:25 | |
| *** JayJ has quit IRC | 21:25 | |
| *** e0ne has joined #openstack-infra | 21:25 | |
| *** wenlock has quit IRC | 21:26 | |
| *** JayJ has joined #openstack-infra | 21:26 | |
| *** wenlock has joined #openstack-infra | 21:26 | |
| *** shakamun_ has quit IRC | 21:26 | |
| *** shakamunyi_ has quit IRC | 21:27 | |
| nibalizer | fungi: if hiera is set up | 21:28 |
| nibalizer | big if | 21:29 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Add plugins_info to module registry object. https://review.openstack.org/132927 | 21:29 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: (MAINT) Reorganize tests/cmd https://review.openstack.org/134055 | 21:29 |
| mordred | krotscheck: nope | 21:29 |
| mordred | krotscheck: perf concern would be if it was going from varchar(65k) to text() | 21:29 |
| krotscheck | fungi: ^^ | 21:29 |
| mordred | since tose are stored differencly | 21:29 |
| krotscheck | Gotcha | 21:29 |
| mordred | but text and mediumtext are implemeneted identically | 21:29 |
| *** shakamunyi has joined #openstack-infra | 21:30 | |
| fungi | krotscheck: mordred: got ot | 21:30 |
| fungi | it | 21:30 |
| mordred | btw - can we call the infra group "infra" and not "openstack-ci" ? | 21:30 |
| mordred | or something | 21:31 |
| krotscheck | mordred: Yes. That’s all config-ish. | 21:31 |
| mordred | woot | 21:31 |
| fungi | mordred: i think that's transitional since it also currently is used by jeepyb to know where to update bugs | 21:31 |
| *** baoli has quit IRC | 21:31 | |
| fungi | mordred: so i would expect to update it once we're done migrating our projects off lp | 21:31 |
| openstackgerrit | Merged openstack-infra/system-config: Use openjdk-1.8 packages for Fedora 21 https://review.openstack.org/132757 | 21:31 |
| krotscheck | So all we still need is one more +2 on https://review.openstack.org/#/c/134039/2 | 21:31 |
| *** achanda has quit IRC | 21:32 | |
| *** AlexF has quit IRC | 21:33 | |
| *** armax has quit IRC | 21:34 | |
| openstackgerrit | Merged openstack-infra/system-config: Don't use -o argument to "free" https://review.openstack.org/132923 | 21:35 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Add plugins_info to module registry object. https://review.openstack.org/132927 | 21:36 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: (MAINT) Reorganize tests/cmd https://review.openstack.org/134055 | 21:36 |
| *** Viswanath has joined #openstack-infra | 21:36 | |
| *** dimtruck is now known as zz_dimtruck | 21:38 | |
| *** marun has quit IRC | 21:39 | |
| *** Viswanath has quit IRC | 21:40 | |
| hashar | zaro: I have followed up on https://bugs.launchpad.net/python-jenkins/+bug/1177831 | 21:41 |
| uvirtbot | Launchpad bug 1177831 in python-jenkins "build_job with parameters fails since POST is not used." [Undecided,In progress] | 21:41 |
| openstackgerrit | Merged openstack-infra/storyboard: Clean recovery for import. https://review.openstack.org/134039 | 21:41 |
| openstackgerrit | Merged openstack-infra/storyboard: Updated size of comment content. https://review.openstack.org/134051 | 21:41 |
| *** Viswanath has joined #openstack-infra | 21:41 | |
| mordred | krotscheck: ^^ | 21:42 |
| hashar | zaro: and yeah maybe porting python-jenkins to stack forge was not a good idea. Might want to switch everything to https://pypi.python.org/pypi/jenkinsapi | 21:42 |
| fungi | krotscheck: once those ^ show up on the prod server, i'll try again | 21:42 |
| krotscheck | mordred: Awesooome | 21:42 |
| fungi | awezoom | 21:43 |
| mordred | hashar: yah. looks maintained | 21:43 |
| krotscheck | fungi: So 15 minutes. | 21:43 |
| krotscheck | Coolio | 21:43 |
| hashar | mordred: though "our" python-jenkins received lot of love and is under the openstack umbrella :D | 21:43 |
| *** annegent_ has quit IRC | 21:44 | |
| *** dmellado_ has joined #openstack-infra | 21:44 | |
| openstackgerrit | Merged openstack-infra/system-config: Initial Fedora 21 nodepool disk-image creation https://review.openstack.org/132925 | 21:44 |
| *** Viswanath has quit IRC | 21:44 | |
| mordred | hashar: indeed :) | 21:45 |
| zaro | hashar: thanks for confirming. | 21:47 |
| hashar | zaro: I will get Wikimedia Jenkins upgraded and see what happens :D | 21:48 |
| *** hdd has quit IRC | 21:48 | |
| *** dmellado_ has quit IRC | 21:48 | |
| zaro | hashar: just as we are thinking about using jenkinsapi, pelix is working on adding folder plugin support to python-jenkins :) | 21:49 |
| zaro | hashar: https://review.openstack.org/133300 | 21:49 |
| zaro | i really should tell pelix to check jenkinsapi to see if folder plugin support is there. | 21:50 |
| hashar | zaro: can ask on the change | 21:51 |
| zaro | hashar: ahh of course. | 21:51 |
| *** wenlock has quit IRC | 21:52 | |
| jogo | what is the procedure for handling bad 3rd party votes? http://huawei-openstack-ci.huawei.com/dsvm-tempest-huawei-sds/371/console.html | 21:53 |
| fungi | jogo: ask the operator to correct/remove them, or ignore them | 21:54 |
| krotscheck | fungi: Looks like it merged. | 21:54 |
| zaro | hashar: when i goto my jenkins http://host:8080/job/myjob/buildWithParameters i either get a blank page or an job is not-parameterized error. do you get something that says you need to do a POST? | 21:54 |
| hashar | zaro: yeah the web form says I need to POST | 21:55 |
| *** tjones has left #openstack-infra | 21:55 | |
| hashar | zaro: but doing a GET over the api works fine :/ | 21:55 |
| *** wenlock has joined #openstack-infra | 21:55 | |
| fungi | jogo: and if widespread, request disablement of voting for their ci (and if outright broken, ask for their ci service account to be temporarily deactivated) | 21:55 |
| *** annegent_ has joined #openstack-infra | 21:55 | |
| krotscheck | fungi: I’m going to go wandering in search of fud. Also food, but with all the advertising around here I’m more likely to find Fud. | 21:55 |
| jogo | fungi: should I email them and CC some ML? | 21:55 |
| jogo | versus private email | 21:55 |
| fungi | krotscheck: cast iron stomach ftw | 21:55 |
| fungi | jogo: Huawei ML2 CI or Huawei volume CI? | 21:56 |
| jogo | fungi: funny enough Huawei ML2 CI check is running on cinder | 21:57 |
| jogo | and running dsvm-tempest-huawei-sds | 21:58 |
| *** e0ne has quit IRC | 21:58 | |
| fungi | jogo: neither of them seem to be in https://review.openstack.org/#/admin/groups/91,members so presumably not leaving actual verify votes, just comments? | 21:58 |
| jogo | fungi: patrickeast already pinged them | 21:59 |
| zaro | hashar: hmmm, odd. i just get a blank page, but the job does starts. wondering if maybe i don't have security turned on? | 21:59 |
| fungi | jogo: patrickeast: thanks! | 21:59 |
| jogo | fungi: that is correct | 21:59 |
| *** dustins has quit IRC | 21:59 | |
| jogo | fungi: not voting | 21:59 |
| hashar | zaro: no idea :/ | 21:59 |
| fungi | jogo: patrickeast: if you feel like they're not responsive, give me a heads up and i'll disable the account temporarily to contain further noise | 21:59 |
| *** achanda has joined #openstack-infra | 22:00 | |
| jogo | fungi: well for one thing, they are using a ML2 (networking?) account for cinder | 22:00 |
| jogo | which IMHO is reason enough | 22:00 |
| jogo | but I am not a cinder person, so don't take my word for it | 22:00 |
| fungi | jogo: yeah, that seems incorrect to me too :/ | 22:00 |
| *** zz_dimtruck is now known as dimtruck | 22:00 | |
| jogo | jgriffith patrickeast: ^ | 22:00 |
| *** mattfarina has quit IRC | 22:01 | |
| fungi | patrickeast: how long ago did you reach out to them? and is it still continuing to comment? | 22:02 |
| patrickeast | jogo: fungi: imo they shouldn’t be reporting failures for so long.. I emailed them maybe 4 hours ago | 22:02 |
| *** mfink has joined #openstack-infra | 22:02 | |
| patrickeast | but it has been failing on my reviews for a couple weeks now.. | 22:02 |
| fungi | yeah, that's quite a long time for them to not notice | 22:03 |
| *** annegent_ has quit IRC | 22:03 | |
| patrickeast | it doesn’t do anything but cause noise now, might as well disable it until they fix it | 22:03 |
| fungi | i'll go ahead and deactivate the account and post a notice to the third-party-announce ml | 22:03 |
| jogo | patrickeast: makes sense to me | 22:03 |
| jogo | fungi: thanks | 22:03 |
| zaro | hashar: want to add the comment on https://review.openstack.org/#/c/106314/ ? | 22:04 |
| *** amcrn has quit IRC | 22:04 | |
| *** achanda has quit IRC | 22:06 | |
| hashar | zaro: git blame! :-D | 22:07 |
| hashar | zaro: but yeah I agree a one-liner comment will help | 22:07 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Add plugins_info to module registry object. https://review.openstack.org/132927 | 22:08 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: (MAINT) Reorganize tests/cmd https://review.openstack.org/134055 | 22:08 |
| krotscheck | fungi: I lied. NOW I’m going out for food. | 22:09 |
| openstackgerrit | Antoine Musso proposed stackforge/python-jenkins: Yield HTTPError reason on open error https://review.openstack.org/106314 | 22:10 |
| hashar | zaro: {done}. Thanks | 22:10 |
| fungi | krotscheck: no worries... been rerunning for 10+ minutes now and hasn't dumped out yet | 22:10 |
| mordred | fungi: woot | 22:10 |
| fungi | mordred: we very well may be completely on sb (for infra) before the day is out! | 22:11 |
| mordred | fungi: that's so sexy | 22:11 |
| fungi | mordred: yes, i know. we operate a sexy, sexy project | 22:12 |
| fungi | now to finish telling some random ci they're broken | 22:12 |
| mordred | mmm. | 22:12 |
| fungi | patrickeast: jogo: have a good example change i can link in the list e-mail? | 22:13 |
| patrickeast | fungi: https://review.openstack.org/#/c/132370/ has a history of several failures from them with different causes | 22:13 |
| *** achanda has joined #openstack-infra | 22:14 | |
| hashar | fungi: iirc storyboard does not handle emails notification yet right? | 22:15 |
| mordred | hashar: that's right | 22:15 |
| fungi | krotscheck: when you get back... the migrate script completed without error, and imported another 34 bugs, but that's nowhere near as many as i expected | 22:15 |
| mordred | hashar: I believe it's next on the list | 22:15 |
| mordred | fungi: hrm. that's annoying | 22:15 |
| fungi | hashar: yeah, as noted in the design summit session infra will live without it for our projects in the short term understanding that it's "soon come" | 22:15 |
| fungi | patrickeast: thanks! | 22:16 |
| hashar | fungi: mordred: when the room was asked whether mail was a blocker, only a couple folks (including me) voted. So I guess it is all fine | 22:16 |
| fungi | hashar: yep, sorry. as usual the minority is trodden upon by the uncaring majority ;) | 22:17 |
| zaro | i also voted +1 for mail support | 22:17 |
| hashar | fungi: yeah consensus driven. And I am not that active in openstack bugs anyway so I have voted merely for information | 22:17 |
| fungi | zaro: +1 for not migrating infra projects until we had mail support you mean? | 22:17 |
| zaro | yes | 22:18 |
| *** gokrokve has quit IRC | 22:18 | |
| *** kgiusti has quit IRC | 22:18 | |
| fungi | zaro: notably, we haven't entirely pulled the trigger yet. the import process is incremental and we've only merged documentation changes to a handful of our projects so far encouraging users to report bugs on sb | 22:18 |
| zaro | i'm still updating lp bugs otherwise the person who entered the bug would never get the updates. | 22:18 |
| fungi | zaro: fair point | 22:19 |
| * fungi is the cavalier "break a few eggs to make an omelet" sort | 22:20 | |
| * hashar picks bacon | 22:20 | |
| fungi | excellent choice | 22:23 |
| fungi | one can never go wrong with bacon | 22:23 |
| mordred | mmm. bacon | 22:23 |
| *** alex7376 has joined #openstack-infra | 22:24 | |
| *** cpowell has quit IRC | 22:24 | |
| krotscheck | hashar: Working on that | 22:24 |
| krotscheck | fungi: Well, it ran for a while before erroring out last time, no? | 22:24 |
| fungi | krotscheck: since you're clearly not actually eating after all, who do i talk to about storyboard logging me out after a few hours? | 22:25 |
| fungi | krotscheck: ran _and_ completed | 22:25 |
| krotscheck | fungi: I’m at Blue Moon Burgers right now, they have the wifi | 22:25 |
| hashar | krotscheck: "Antoine" here, I gave you a short presentation of Phabricator. Really appreciated your session moderation kudos! | 22:25 |
| krotscheck | fungi: I think the person to talk to is me. | 22:25 |
| fungi | krotscheck: dammit man, eat your burgers and get back to us later | 22:25 |
| krotscheck | hashar: Yes! I remember you, mostly because it was your user account that I had to fix a bug for :) | 22:25 |
| hashar | krotscheck: has it been figured out or will you need more details? I am happy to help however I can, just drop me an email ! | 22:26 |
| hashar | and definitely focus on that burger | 22:26 |
| mordred | fungi: I found a thing better than bacon though | 22:26 |
| fungi | mordred: lies | 22:26 |
| *** asselin_ has joined #openstack-infra | 22:26 | |
| mordred | fungi: poulet en vessie, a chicken with truffle-stuffed skin cooked inside a pig's bladder in court bouillon, finished with a sauce of truffles, foie gras, and cream | 22:26 |
| krotscheck | hashar: It’s been fixed. | 22:26 |
| *** asettle has quit IRC | 22:26 | |
| krotscheck | fungi: THe logging out thing is an annoying race condition bug in the refresh token logic of the webclient | 22:27 |
| *** alex7376 is now known as asettle | 22:27 | |
| fungi | mordred: that sounds far too complicated. shave the meat off a pig's back and fry it in a pan. simplicity counts for something | 22:27 |
| *** miqui has quit IRC | 22:27 | |
| lifeless | fungi: take pig, thrown in fire. Done. | 22:27 |
| fungi | lifeless: yes. a thousand times yes | 22:27 |
| mordred | fungi: on that note - did I tell you about teh pan-seared foie gras I had? it was actually just the whole liver seared in a pan | 22:28 |
| *** baoli has joined #openstack-infra | 22:28 | |
| fungi | mordred: no, but as a confessed carnivore, sounds lovely | 22:28 |
| *** dannywilson has quit IRC | 22:29 | |
| mordred | fungi: it was ... VERY good | 22:29 |
| fungi | mordred: i was just glad to be somewhere i could get good marinated raw beef for a change. the tartare and carpaccio in paris... excellence | 22:29 |
| mordred | ++ | 22:29 |
| hashar | mordred: that stuffed chicken is deadly. | 22:29 |
| mordred | hashar: SO GOOD | 22:29 |
| mordred | hashar: sandy and I stopped by the bocuse restaurant in lyon | 22:29 |
| hashar | A+1 | 22:29 |
| mordred | hashar: my blood is still fatty | 22:29 |
| hashar | France make you fat very easily if you like food | 22:30 |
| hashar | that is why we drink wine as well | 22:30 |
| krotscheck | fungi: So, I think the migration is done? | 22:30 |
| fungi | krotscheck: the migration script thinks so. i worry it did too little | 22:30 |
| *** MarkAtwood has quit IRC | 22:30 | |
| *** asselin_ has quit IRC | 22:30 | |
| fungi | krotscheck: someone who isn't me might want to pick random older closed openstack-ci bugs and spot-check, when not burgering | 22:31 |
| hashar | zaro: will give a look at the other jjb patches later in the week :) | 22:31 |
| hashar | for now *wave* | 22:31 |
| krotscheck | fungi: That’s what I’m looking | 22:31 |
| zaro | hashar: bouncer | 22:31 |
| *** dkranz has quit IRC | 22:31 | |
| *** dannywilson has joined #openstack-infra | 22:31 | |
| hashar | zaro: yeah gotta set one up. I promise | 22:32 |
| waynr | zaro: JJB plugins_info stuff is ready for review | 22:33 |
| *** pc_m has quit IRC | 22:33 | |
| krotscheck | fungi: So, I have 189 open stories, 836 merged ones, and 291 invalid ones in system-config | 22:33 |
| fungi | krotscheck: i'm reaching the time of day where my diet steadily trends toward rum, so i'll take your word for it. that sounds reasonably large | 22:33 |
| *** andreykurilin_ has quit IRC | 22:34 | |
| *** patrickeast has quit IRC | 22:34 | |
| krotscheck | fungi: That seems like a large enough number? | 22:34 |
| *** sabeen1 has quit IRC | 22:34 | |
| fungi | and also reversing my -2 on the system-config documentation change | 22:34 |
| *** sabeen2 has joined #openstack-infra | 22:34 | |
| SpamapS | hm | 22:34 |
| *** patrickeast has joined #openstack-infra | 22:35 | |
| SpamapS | http://paste.ubuntu.com/8973027/ | 22:35 |
| *** tonytan4ever has quit IRC | 22:35 | |
| SpamapS | anybody know why I get failure to install pbr on a tox run? | 22:35 |
| SpamapS | when I can verify pbr is already installed in that venv | 22:36 |
| SpamapS | ? | 22:36 |
| mordred | SpamapS: looking | 22:36 |
| *** mbacchi has quit IRC | 22:36 | |
| fungi | SpamapS: that looks like a pip problem | 22:36 |
| *** shaps has joined #openstack-infra | 22:36 | |
| fungi | SpamapS: is it consistently reproducible? | 22:37 |
| SpamapS | virtualenv 1.11.6 , too ancient? | 22:37 |
| SpamapS | fungi: yes | 22:37 |
| SpamapS | seems systemic, I can't run tox on anything | 22:37 |
| fungi | SpamapS: that should be plenty new enough | 22:37 |
| *** shaps has quit IRC | 22:37 | |
| SpamapS | tox 1.7.2 | 22:37 |
| fungi | SpamapS: broken network? | 22:38 |
| SpamapS | fungi: I can pip install anything | 22:38 |
| fungi | SpamapS: you can pip install anything but can't run tox on anything... | 22:38 |
| SpamapS | fungi: right | 22:38 |
| *** _shaps_ has joined #openstack-infra | 22:38 | |
| * fungi is reasonably stumped | 22:39 | |
| *** _shaps_ has quit IRC | 22:39 | |
| SpamapS | FTR, I don't have distro setuptools installed | 22:39 |
| fungi | SpamapS: so it's not _just_ gerrit-dash-creator triggering the issue | 22:40 |
| fungi | ? | 22:40 |
| SpamapS | fungi: correct | 22:40 |
| SpamapS | fungi: anything that uses pbr | 22:40 |
| SpamapS | fungi: os-apply-config .. heat.. anything | 22:40 |
| *** camunoz has joined #openstack-infra | 22:41 | |
| fungi | SpamapS: just on that particular machine, or is there a way i can reproduce? | 22:41 |
| *** MarkAtwood has joined #openstack-infra | 22:41 | |
| SpamapS | fungi: just here, so I'm sure it's some weird thing in my local python | 22:41 |
| *** marun has joined #openstack-infra | 22:41 | |
| SpamapS | http://paste.ubuntu.com/8973177/ | 22:42 |
| *** armax has joined #openstack-infra | 22:42 | |
| SpamapS | pip freeze output | 22:42 |
| zaro | waynr: will take a look later, please add hashar as a reviewer | 22:42 |
| *** dimtruck is now known as zz_dimtruck | 22:42 | |
| fungi | fwiw that's current released pbr | 22:43 |
| SpamapS | fungi: note that I've tried now with tox 1.8.1, and 1.6.1, just for bisecting purposes | 22:43 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/git-review: Switched documentation to point to storyboard https://review.openstack.org/133847 | 22:44 |
| SpamapS | fungi: happens if I just manually run $venvpath/bin/pip install -r test-requirements.txt -r requirements.txt | 22:44 |
| SpamapS | File "/home/clint/src/heat/.tox/py27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 589, in easy_install | 22:45 |
| SpamapS | raise DistutilsError(msg) | 22:45 |
| SpamapS | I thought pip didn't use easy_install ? | 22:45 |
| *** dmellado_ has joined #openstack-infra | 22:45 | |
| fungi | SpamapS: pip doesn't, but pip causes setup.py of installing projects to run, and any which declare a setup_requires will use easy_install | 22:46 |
| mordred | for setup_rqeuires things it gets triggered | 22:46 |
| SpamapS | ah | 22:46 |
| SpamapS | Downloading/unpacking hacking>=0.8.0,<0.9 (from -r test-requirements.txt (line 6)) | 22:46 |
| SpamapS | Downloading hacking-0.8.1.tar.gz (51kB): 51kB downloaded | 22:46 |
| fungi | SpamapS: it's the legacy of python packaging hate | 22:46 |
| SpamapS | Running setup.py (path:/home/clint/src/heat/.tox/py27/build/hacking/setup.py) egg_info for package hacking | 22:46 |
| SpamapS | Couldn't find index page for 'pbr' (maybe misspelled?) | 22:46 |
| SpamapS | hence that | 22:46 |
| *** baoli has quit IRC | 22:47 | |
| *** baoli has joined #openstack-infra | 22:47 | |
| fungi | SpamapS: i'm going to bet that there's something broken with the pbr as installed in your environment. wild guess | 22:47 |
| mordred | I'm going to guess something with the setuptools or venv, actually | 22:47 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Switched documentation to point to storyboard. https://review.openstack.org/133864 | 22:48 |
| fungi | something causing it not to find the egg_info for pbr though | 22:48 |
| mordred | I'd do a pip uninstall of setuptools, then install it again, then install pip again (both with -U) | 22:48 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Switched documentation to point to storyboard. https://review.openstack.org/133851 | 22:48 |
| mordred | if it was me | 22:48 |
| fungi | python packaging is still so unpleasantly fiddly | 22:49 |
| SpamapS | Installing collected packages: setuptools Found existing installation: setuptools 6.0.2 | 22:49 |
| SpamapS | that looks.. evil? | 22:49 |
| *** koolhead17 has joined #openstack-infra | 22:50 | |
| *** dmellado_ has quit IRC | 22:50 | |
| SpamapS | mordred: no dice | 22:50 |
| mordred | SpamapS: hrm. something seems VERY borked on your system | 22:50 |
| mordred | SpamapS: I recommend using go | 22:50 |
| fungi | mordred: we should rewrite the python interpreter in go | 22:51 |
| mordred | SpamapS: you said you can't do this anywhere, right? | 22:51 |
| SpamapS | mordred: on a fresh 14.04 box no, this doesn't happen | 22:51 |
| fungi | mordred: i think he said he can't reproduce anywhere except on his sustem | 22:51 |
| fungi | system | 22:51 |
| fungi | sustemance | 22:52 |
| SpamapS | Willing to torch /usr/local/lib/python2.7 if you think it may help | 22:52 |
| krotscheck | fungi: So, there’s 1325 bugs in LP-openstack-ci. There are 1316 in storyboard, meaning we’re missing 9. I’m off to go look for those. | 22:52 |
| *** hashar has quit IRC | 22:53 | |
| fungi | krotscheck: take a sweater | 22:53 |
| mordred | SpamapS: I mean, you have this problem in multiple directories and multiple virtualenvs, right? | 22:53 |
| mordred | SpamapS: or is ti just in that one directory? | 22:53 |
| fungi | krotscheck: it's <1% but i agree that's worth figuring out | 22:53 |
| SpamapS | mordred: all | 22:54 |
| mordred | SpamapS: hrm. I have no idea why a fresh virtualenv would get borked bby something | 22:54 |
| SpamapS | ok so I just torched /usr/local/lib/python2.7, and re-installed pip using get-pip.py .. and then pip installed virtualenv | 22:54 |
| mordred | yah | 22:54 |
| SpamapS | fail | 22:55 |
| mordred | do you have python-virtualenv installed? | 22:55 |
| SpamapS | no | 22:55 |
| mordred | or python-pip | 22:55 |
| SpamapS | no | 22:55 |
| mordred | WTF | 22:55 |
| mordred | SpamapS: dude. that's really impressive | 22:55 |
| SpamapS | mordred: do I get the trophy back? :) | 22:55 |
| *** cnesa has quit IRC | 22:55 | |
| mordred | any chance you have a proxy/mirror stting pointing to something bizarre? | 22:55 |
| SpamapS | mordred: disabled "just in case" | 22:55 |
| dmsimard | Hey, any clue if lodgeit.py (https://raw.githubusercontent.com/openstack-infra/lodgeit/master/scripts/lodgeit.py) should be able to talk with paste.openstack.org ? Doesn't seem like I can get it to work. | 22:56 |
| dmsimard | My bad, here's the human link: https://github.com/openstack-infra/lodgeit/blob/master/scripts/lodgeit.py | 22:56 |
| dmsimard | Oh shit, just noticed that thing is 4 years old | 22:57 |
| mordred | yah. lodgeit is long in the tooth | 22:57 |
| mordred | pastebinit does | 22:57 |
| *** thedodd has quit IRC | 22:58 | |
| mordred | pastebinit -b http://paste.openstack.org works - and it's possible to configure pastebinit to use paste.o.o as the default, but I forget how | 22:58 |
| dmsimard | mordred: Thanks, will look at it. | 22:59 |
| *** koolhead17 has quit IRC | 22:59 | |
| *** koolhead17 has joined #openstack-infra | 23:00 | |
| *** koolhead17 has quit IRC | 23:00 | |
| *** koolhead17 has joined #openstack-infra | 23:00 | |
| SpamapS | mordred: ~/.pastebinit.xml | 23:01 |
| *** wenlock has quit IRC | 23:02 | |
| SpamapS | mordred: if I manually pip install pbr in the venv, then hacking installs fine | 23:03 |
| *** wenlock has joined #openstack-infra | 23:03 | |
| *** dizquierdo has quit IRC | 23:04 | |
| *** koolhead17 has quit IRC | 23:04 | |
| dstufft | oh | 23:04 |
| SpamapS | actually I can manually just pip install hacking | 23:04 |
| dstufft | I need to see about stealing setup_requires control away from setuptools | 23:05 |
| dstufft | setuptools shouldn't go touching the network, that's naughty | 23:05 |
| *** xyang0 has quit IRC | 23:05 | |
| fungi | dstufft: yes, pip needs to reign in the naughty | 23:05 |
| fungi | please with the making | 23:06 |
| SpamapS | you all are cute but this is absolutely maddening | 23:06 |
| fungi | SpamapS: it's pretty well all maddening. hence we're all now stark raving mad | 23:07 |
| *** JayJ has quit IRC | 23:08 | |
| fungi | SpamapS: so anything with a requirements/test-requirements entry for hacking breaks as described, but hacking directly is fine? | 23:08 |
| dstufft | it looks like setuptools couldn't fetch the pbr page from pypi | 23:08 |
| SpamapS | ok | 23:09 |
| SpamapS | so I know whats going on now | 23:09 |
| SpamapS | and you all, please don't shake me | 23:09 |
| dstufft | do you have a ~/.pydistutils.cfg | 23:09 |
| * fungi cradles head in hands | 23:09 | |
| SpamapS | I _added_ a proxy | 23:09 |
| camunoz | AJaeger: what is the best way to share the Zanata ansible scripts with you? | 23:09 |
| SpamapS | to see if I was missing something | 23:09 |
| SpamapS | GET http://pypi.openstack.org/openstack/ HTTP/1.1 | 23:09 |
| SpamapS | dddddooohhhhhhhh | 23:09 |
| fungi | SpamapS: okay, so pip and easy_install were dealing with the proxy differently i guess | 23:09 |
| SpamapS | fungi: precisely | 23:10 |
| SpamapS | dstufft: BINGO | 23:10 |
| fungi | SpamapS: also, pypi.openstack.org is dead-ish, not updated, and on its way into the wastebin btw | 23:10 |
| SpamapS | $ cat .pydistutils.cfg | 23:10 |
| SpamapS | [easy_install] | 23:10 |
| SpamapS | index_url = http://pypi.openstack.org/openstack/ | 23:10 |
| SpamapS | Because, you know.. thats the fast one! | 23:10 |
| SpamapS | ;) | 23:10 |
| *** Sukhdev has joined #openstack-infra | 23:10 | |
| SpamapS | would be nice if that error showed up _anywhere_ the user could see it. | 23:11 |
| *** amitgandhinz has quit IRC | 23:11 | |
| SpamapS | like "hey your mirror is 404'ing on stuff you need" not "FOAD that doesn't exist." | 23:11 |
| *** baoli has quit IRC | 23:11 | |
| *** amcrn has joined #openstack-infra | 23:12 | |
| *** david-lyle is now known as david-lyle_afk | 23:12 | |
| fungi | also, i totes need a custom 404 page that _actually_ says foad | 23:12 |
| *** baoli has joined #openstack-infra | 23:12 | |
| * fungi takes notes | 23:12 | |
| dstufft | It wouldn't be python packaging if we were nice to our users and told them the information they needed to fix the problem they are encountering | 23:13 |
| *** salv-orlando has quit IRC | 23:14 | |
| fungi | and now comes the point in the evening where i transition from ranting pointlessly to avoiding computers for a bit before i fall unconscious | 23:14 |
| *** MarkAtwood has quit IRC | 23:15 | |
| *** salv-orlando has joined #openstack-infra | 23:15 | |
| SpamapS | http://thebest404pageever.com/ | 23:15 |
| SpamapS | warning: SEIZURE DANGER | 23:15 |
| *** baoli has quit IRC | 23:15 | |
| *** ChuckC has quit IRC | 23:16 | |
| *** baoli has joined #openstack-infra | 23:16 | |
| SpamapS | wow this one is actually pretty awesome | 23:17 |
| SpamapS | http://www.bluedaniel.com/404.shtml | 23:17 |
| fungi | umm... saotome genma playing the keyboard?!? | 23:18 |
| fungi | i have seen too much | 23:19 |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool: Set useful PATH before starting scripts https://review.openstack.org/131921 | 23:19 |
| *** weshay has quit IRC | 23:23 | |
| *** sunil_ has joined #openstack-infra | 23:23 | |
| openstackgerrit | Merged stackforge/python-jenkins: Add a readme and make it appear on pypi.python.org https://review.openstack.org/132099 | 23:24 |
| arosen1 | fungi: any idea about this error from dib? http://paste.openstack.org/show/132569/ | 23:25 |
| *** baoli has quit IRC | 23:25 | |
| fungi | arosen1: pretty vague as errors go, but i know clarkb ran into issues with filling up the disk. make sure you have _lots_ of space? | 23:27 |
| *** Sukhdev has quit IRC | 23:27 | |
| arosen1 | fungi: lolz yup that was the first thing i hit originally and i create a new image with a large disk | 23:28 |
| arosen1 | /dev/mapper/ubuntu--vg-root 95G 14G 77G 15% / | 23:28 |
| ianw | arosen1: yeah, TMP_DIR to a tmpfs can fill up and cause weird issues | 23:28 |
| arosen1 | still a good amount free. | 23:28 |
| fungi | arosen1: yep, that looks fine then. not sure what to suggest other than maybe stuffing more debugging in around that exception to explain the why rather than the what | 23:29 |
| fungi | maybe permissions? seems unlikely though | 23:29 |
| *** rmcall has quit IRC | 23:29 | |
| *** marun has quit IRC | 23:31 | |
| fungi | okay, making myself scarce for the night. later all | 23:31 |
| ianw | arosen1: also, the real error is probably about 50 lines before what's in the paste-bin :) looks like just caught the cleanup there | 23:31 |
| *** mmaglana has joined #openstack-infra | 23:32 | |
| *** patrickeast has quit IRC | 23:32 | |
| *** mmaglana_ has quit IRC | 23:32 | |
| arosen1 | ianw: 2014-11-12 14:58:54,572 INFO nodepool.image.build.devstack-trusty-dib: sfdisk: ERROR: sector 0 does not have an msdos signature | 23:32 |
| arosen1 | ? | 23:32 |
| *** dannywilson has quit IRC | 23:33 | |
| *** patrickeast has joined #openstack-infra | 23:33 | |
| ianw | arosen1: is your base .qcow2 ok? sounds like maybe a corrupt or truncated file? | 23:33 |
| arosen1 | http://paste.openstack.org/show/132570/ | 23:33 |
| *** dannywilson has joined #openstack-infra | 23:34 | |
| krotscheck | fungi: So, the impacted bugs that I’ve found are ones that have an ‘affects project’ # of > 1. | 23:34 |
| krotscheck | fungi: Which basically means that they’re in there, they were just assigned to only one project. | 23:34 |
| krotscheck | Here’s an example: https://bugs.launchpad.net/openstack-ci/+bug/1012730 | 23:35 |
| uvirtbot | Launchpad bug 1012730 in zuul "No way to tell when a change needs reapproval" [High,Fix released] | 23:35 |
| krotscheck | vs: https://storyboard.openstack.org/#!/story/1012730 | 23:35 |
| ianw | arosen1: /usr/local/bin/../share/diskimage-builder/lib/img-functions: line 119: qemu-img: command not found | 23:36 |
| *** ZZelle_ has quit IRC | 23:37 | |
| arosen1 | ianw: ah thanks good catch. | 23:37 |
| arosen1 | guess we need to add qemu-utils to the readme as well for what to install. | 23:37 |
| arosen1 | would be nice if we logged that as an error rather than info | 23:37 |
| *** peddamat has joined #openstack-infra | 23:38 | |
| ianw | arosen1: becoming more and more convinced nodepool needs a thread to output a nice html dashboard that shows the status of everything | 23:38 |
| *** peddamat has quit IRC | 23:38 | |
| ianw | we've had images not building for too long because nobody was really watching the logs in detail | 23:38 |
| *** peddamat has joined #openstack-infra | 23:38 | |
| *** adalbas has joined #openstack-infra | 23:39 | |
| *** ryanpetrello has quit IRC | 23:39 | |
| *** baoli has joined #openstack-infra | 23:39 | |
| *** marun has joined #openstack-infra | 23:39 | |
| openstackgerrit | Aaron Rosen proposed openstack-infra/nodepool: libffi-dev/libssl-dev/qemu-utils are required https://review.openstack.org/125515 | 23:39 |
| *** ChuckC has joined #openstack-infra | 23:40 | |
| ianw | arosen1: yeah, have you successfully done a build with project-config:tools/build-image.sh? i'd suggest getting that working first, before trying to debug under nodepool | 23:40 |
| arosen1 | ianw: running that now. | 23:41 |
| arosen1 | ianw: can you just run that file directly? | 23:42 |
| arosen1 | it seems to fail with: Can not determine RELEASE | 23:42 |
| *** prad has quit IRC | 23:42 | |
| *** armax has quit IRC | 23:44 | |
| *** marun has quit IRC | 23:44 | |
| *** dmsimard is now known as dmsimard_away | 23:44 | |
| *** signed8b_ has joined #openstack-infra | 23:44 | |
| ianw | arosen1: you may need DIB_RELEASE=trusty for ubuntu | 23:44 |
| * ianw is usually building fedora/centos images :) | 23:44 | |
| *** signed8bit has quit IRC | 23:46 | |
| *** doug-fish has left #openstack-infra | 23:49 | |
| *** baoli has quit IRC | 23:50 | |
| *** baoli has joined #openstack-infra | 23:51 | |
| *** oda-g_ has joined #openstack-infra | 23:53 | |
| *** teran has joined #openstack-infra | 23:53 | |
| *** dims_ has joined #openstack-infra | 23:54 | |
| *** zns has quit IRC | 23:56 | |
| *** gokrokve has joined #openstack-infra | 23:56 | |
| *** amotoki has joined #openstack-infra | 23:56 | |
| *** dims has quit IRC | 23:57 | |
| *** shakamunyi has quit IRC | 23:58 | |
| oda-g_ | Hi all, | 23:58 |
| *** shakamunyi has joined #openstack-infra | 23:58 | |
| oda-g_ | Our CI become failed very recently in execution of devstack. | 23:58 |
| oda-g_ | openstack project create admin -f value -c id | 23:58 |
| oda-g_ | ERROR: openstack The plugin token_endpoint could not be found | 23:59 |
| oda-g_ | Does anyone give me a hint to solve the problem ? | 23:59 |
| patrickeast | oda-g_: oo mine has the same problem | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!