| *** tosky has quit IRC | 00:01 | |
| *** dychen has joined #openstack-infra | 00:04 | |
| *** dychen has quit IRC | 00:08 | |
| *** ianychoi has joined #openstack-infra | 00:14 | |
| *** zbr5 has joined #openstack-infra | 00:22 | |
| *** zbr has quit IRC | 00:22 | |
| *** zbr5 is now known as zbr | 00:22 | |
| *** ramishra has quit IRC | 00:38 | |
| *** ramishra has joined #openstack-infra | 00:40 | |
| *** kevinz has joined #openstack-infra | 01:03 | |
| *** jamesmcarthur has quit IRC | 01:13 | |
| *** jamesmcarthur has joined #openstack-infra | 01:15 | |
| *** icey has quit IRC | 01:15 | |
| *** icey has joined #openstack-infra | 01:16 | |
| *** jamesmcarthur has quit IRC | 01:36 | |
| *** jamesmcarthur has joined #openstack-infra | 01:38 | |
| *** dklyle has quit IRC | 01:40 | |
| *** tkajinam has quit IRC | 01:41 | |
| *** tkajinam has joined #openstack-infra | 01:42 | |
| *** jamesmcarthur has quit IRC | 01:43 | |
| *** armax has quit IRC | 01:47 | |
| *** jamesmcarthur has joined #openstack-infra | 01:48 | |
| *** dklyle has joined #openstack-infra | 01:48 | |
| *** armax has joined #openstack-infra | 02:08 | |
| *** tinwood has quit IRC | 02:08 | |
| *** tkajinam has quit IRC | 02:09 | |
| *** tkajinam has joined #openstack-infra | 02:10 | |
| *** armax has quit IRC | 02:10 | |
| *** xinliang has joined #openstack-infra | 02:11 | |
| *** tinwood has joined #openstack-infra | 02:11 | |
| *** zzzeek has quit IRC | 02:30 | |
| *** zzzeek has joined #openstack-infra | 02:31 | |
| *** zzzeek has quit IRC | 02:36 | |
| *** zzzeek has joined #openstack-infra | 02:40 | |
| *** dklyle has quit IRC | 02:45 | |
| *** armax has joined #openstack-infra | 02:48 | |
| *** armax has quit IRC | 02:54 | |
| *** hamalq has quit IRC | 02:56 | |
| *** jamesmcarthur has quit IRC | 02:57 | |
| *** jamesmcarthur has joined #openstack-infra | 02:57 | |
| *** rcernin has quit IRC | 02:57 | |
| *** jamesmcarthur has quit IRC | 03:03 | |
| *** rcernin has joined #openstack-infra | 03:18 | |
| *** rcernin has quit IRC | 03:21 | |
| *** rcernin has joined #openstack-infra | 03:21 | |
| *** ysandeep|away is now known as ysandeep | 03:24 | |
| *** jamesmcarthur has joined #openstack-infra | 03:29 | |
| *** psachin has joined #openstack-infra | 03:33 | |
| *** ricolin_ has joined #openstack-infra | 03:39 | |
| *** sboyron has quit IRC | 03:41 | |
| *** xinliang has quit IRC | 03:53 | |
| *** ricolin_ has quit IRC | 03:54 | |
| *** jamesmcarthur has quit IRC | 04:07 | |
| *** rcernin has quit IRC | 04:35 | |
| *** rcernin has joined #openstack-infra | 04:35 | |
| *** gyee has quit IRC | 05:08 | |
| *** vishalmanchanda has joined #openstack-infra | 05:11 | |
| *** ykarel has joined #openstack-infra | 05:41 | |
| *** rcernin_ has joined #openstack-infra | 05:42 | |
| *** rcernin has quit IRC | 05:42 | |
| *** jamesmcarthur has joined #openstack-infra | 06:04 | |
| *** jamesmcarthur has quit IRC | 06:09 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Move snaps ACL to x https://review.opendev.org/c/openstack/project-config/+/770538 | 07:00 |
|---|---|---|
| openstackgerrit | Merged openstack/project-config master: Create microstack-specs project https://review.opendev.org/c/openstack/project-config/+/770460 | 07:11 |
| *** ralonsoh has joined #openstack-infra | 07:19 | |
| *** zzzeek has quit IRC | 07:28 | |
| *** rcernin_ has quit IRC | 07:28 | |
| *** xinliang has joined #openstack-infra | 07:30 | |
| *** zzzeek has joined #openstack-infra | 07:31 | |
| *** eolivare has joined #openstack-infra | 07:35 | |
| *** jcapitao has joined #openstack-infra | 07:40 | |
| *** openstackgerrit has quit IRC | 07:47 | |
| *** jpena|off is now known as jpena | 07:51 | |
| *** JayF has quit IRC | 07:52 | |
| *** nightmare_unreal has joined #openstack-infra | 07:53 | |
| *** slaweq has joined #openstack-infra | 07:59 | |
| *** diablo_rojo__ has quit IRC | 08:01 | |
| *** hashar has joined #openstack-infra | 08:03 | |
| *** slaweq has quit IRC | 08:04 | |
| *** openstackgerrit has joined #openstack-infra | 08:05 | |
| openstackgerrit | Sorin Sbârnea proposed openstack/project-config master: Move git-review zuul config in-tree https://review.opendev.org/c/openstack/project-config/+/763808 | 08:05 |
| *** rcernin_ has joined #openstack-infra | 08:06 | |
| *** piotrowskim has joined #openstack-infra | 08:07 | |
| *** slaweq has joined #openstack-infra | 08:10 | |
| *** andrewbonney has joined #openstack-infra | 08:13 | |
| openstackgerrit | Sorin Sbârnea proposed openstack/project-config master: Move git-review zuul config in-tree https://review.opendev.org/c/openstack/project-config/+/763808 | 08:21 |
| *** dchen has quit IRC | 08:23 | |
| *** akahat|rover is now known as akahat|lunch | 08:25 | |
| *** rpittau|afk is now known as rpittau | 08:25 | |
| *** rcernin_ has quit IRC | 08:26 | |
| *** sboyron has joined #openstack-infra | 08:27 | |
| *** gfidente|afk is now known as gfidente | 08:32 | |
| *** tosky has joined #openstack-infra | 08:39 | |
| *** xinliang has quit IRC | 08:58 | |
| *** lucasagomes has joined #openstack-infra | 09:05 | |
| *** rfolco has joined #openstack-infra | 09:05 | |
| *** jamesmcarthur has joined #openstack-infra | 09:06 | |
| *** jamesmcarthur has quit IRC | 09:28 | |
| *** derekh has joined #openstack-infra | 09:35 | |
| zbr | any infra-root around to help with ^? | 09:44 |
| *** akahat|lunch is now known as akahat|rover | 10:04 | |
| frickler | zbr: commented, please enlighten me if my understanding is wrong | 10:07 |
| zbr | commented, i hope it explains. | 10:11 |
| zbr | catch-22 between dropping py27 and moving jobs in-tree | 10:12 |
| zbr | both needed/desired/approved | 10:12 |
| frickler | zbr: hmm, o.k., I think you'll still have to combine 770556 and 770539, but we'll see that soon | 10:16 |
| zbr | project-config must be merged first as that prevents testing anything else. | 10:16 |
| frickler | it prevents merging anything else, but shouldn't affect results of the individual tests | 10:18 |
| zbr | tests are already broken | 10:18 |
| zbr | (py27) | 10:18 |
| zbr | so is not affecting anything, look at build history, last success in september | 10:18 |
| zbr | other tests are ok, only py27 is broken. | 10:19 |
| frickler | py36 and pep8 are also failing on 770539, need to combine with the patch to fix those in order to get something mergeable | 10:19 |
| zbr | and for this reason, moving the config includes dropping py27 testing (openstack-python-jobs-no-constraints goes away keeping only tox-pep8 from it) | 10:19 |
| zbr | that templace had only two jobs inside: pep8 and py27. | 10:20 |
| zbr | now I feel sorry that i did not try to convince ajaeger to merge the original version of removal from that morning, the one that was only dropping py27 testing. | 10:20 |
| zbr | it already had one +2 | 10:21 |
| zbr | i doubt anyone involved is against the target: in-tree jobs + supporting only py3.5+ | 10:21 |
| openstackgerrit | Merged openstack/project-config master: Move git-review zuul config in-tree https://review.opendev.org/c/openstack/project-config/+/763808 | 10:24 |
| *** hashar has quit IRC | 10:39 | |
| *** ociuhandu has joined #openstack-infra | 10:40 | |
| *** dtantsur|afk is now known as dtantsur | 10:41 | |
| *** ysandeep is now known as ysandeep|afk | 11:04 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 11:19 | |
| ralonsoh | hello folks | 11:31 |
| ralonsoh | I've seen recently many CI jobs failing due to mirror access problems | 11:31 |
| ralonsoh | e.g.: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_1e5/770356/2/gate/openstack-tox-py36/1e56e46/job-output.txt | 11:32 |
| ralonsoh | 2021-01-13 09:47:15.566438 | ubuntu-bionic | W: Failed to fetch https://mirror.regionone.limestone.opendev.org/ubuntu/dists/bionic-security/InRelease Unable to connect to mirror.regionone.limestone.opendev.org:https: | 11:32 |
| ralonsoh | so far, the mirror failing is mirror.regionone.limestone.opendev.org | 11:33 |
| ralonsoh | I'm checking several failing CI jobs | 11:33 |
| *** ociuhandu has quit IRC | 11:36 | |
| *** jcapitao is now known as jcapitao_lunch | 11:40 | |
| sshnaidm|ruck | ralonsoh, same for us, limestone jobs have a lot of retries | 12:09 |
| *** rlandy has joined #openstack-infra | 12:16 | |
| *** ramishra has quit IRC | 12:19 | |
| zbr | :( | 12:21 |
| *** ramishra has joined #openstack-infra | 12:23 | |
| *** armax has joined #openstack-infra | 12:24 | |
| *** ociuhandu has joined #openstack-infra | 12:45 | |
| *** jpena is now known as jpena|lunch | 12:49 | |
| *** jcapitao_lunch is now known as jcapitao | 12:55 | |
| *** ociuhandu has quit IRC | 12:55 | |
| *** ociuhandu has joined #openstack-infra | 12:56 | |
| *** ociuhandu has quit IRC | 12:56 | |
| *** vishalmanchanda has quit IRC | 13:01 | |
| *** ociuhandu has joined #openstack-infra | 13:01 | |
| *** rlandy is now known as rlandy|mtg | 13:04 | |
| *** ysandeep|afk is now known as ysandeep | 13:25 | |
| *** jamesmcarthur has joined #openstack-infra | 13:26 | |
| *** jamesmcarthur has quit IRC | 13:30 | |
| *** rlandy|mtg is now known as rlandy | 13:44 | |
| *** jpena|lunch is now known as jpena | 13:51 | |
| zbr | frickler: clarkb fungi: https://review.opendev.org/c/opendev/git-review/+/770539 is ready | 14:08 |
| *** d34dh0r53 has quit IRC | 14:10 | |
| *** d34dh0r53 has joined #openstack-infra | 14:13 | |
| *** fresta has quit IRC | 14:18 | |
| *** fresta has joined #openstack-infra | 14:18 | |
| *** vishalmanchanda has joined #openstack-infra | 14:35 | |
| *** jamesmcarthur has joined #openstack-infra | 14:44 | |
| *** hashar has joined #openstack-infra | 14:47 | |
| *** auristor has quit IRC | 14:47 | |
| *** armax has quit IRC | 14:48 | |
| *** armax has joined #openstack-infra | 14:50 | |
| *** auristor has joined #openstack-infra | 14:50 | |
| *** ociuhandu_ has joined #openstack-infra | 14:58 | |
| *** gryf has joined #openstack-infra | 15:00 | |
| *** ociuhandu has quit IRC | 15:02 | |
| *** dwilde has joined #openstack-infra | 15:08 | |
| fungi | ralonsoh: sshnaidm|ruck: frickler rebooted the server after it was offline/hung for some reason so hopefully okay since ~13:00 utc | 15:10 |
| sshnaidm|ruck | fungi, thanks | 15:12 |
| ralonsoh | fungi thanks! | 15:12 |
| sshnaidm|ruck | so far I don't see retries | 15:12 |
| fungi | occasional retries are still expected in limestone, we worked around the fact that some nodes seem to boot with no working ipv4 route by having the base job test for that and discard early any which fail, and those will show up as automatic retries | 15:15 |
| *** psachin has quit IRC | 15:28 | |
| *** ociuhandu_ has quit IRC | 15:35 | |
| *** ociuhandu has joined #openstack-infra | 15:35 | |
| *** ricolin has quit IRC | 15:38 | |
| *** hashar is now known as hasharAway | 15:43 | |
| *** jamesmcarthur has quit IRC | 15:45 | |
| *** dklyle has joined #openstack-infra | 15:46 | |
| *** jamesmcarthur has joined #openstack-infra | 15:46 | |
| *** ricolin has joined #openstack-infra | 15:47 | |
| *** dwilde has quit IRC | 15:49 | |
| *** dwilde has joined #openstack-infra | 15:51 | |
| *** jamesmcarthur has quit IRC | 15:51 | |
| *** jamesmcarthur has joined #openstack-infra | 15:54 | |
| *** dwilde has quit IRC | 15:59 | |
| *** ykarel is now known as ykarel|away | 16:09 | |
| *** ociuhandu_ has joined #openstack-infra | 16:11 | |
| *** ociuhandu has quit IRC | 16:11 | |
| *** ysandeep is now known as ysandeep|out | 16:12 | |
| *** ykarel|away has quit IRC | 16:16 | |
| gryf | hi. I can see repetitive issue with some of the gates on fetching package lists (like here: https://zuul.opendev.org/t/openstack/build/677c550cbc3843849259c1da34532c17/log/controller/logs/devstacklog.txt) | 16:19 |
| clarkb | gryf: did that job run in limestone? | 16:20 |
| clarkb | the mirror there needed to be restarted as it was marked offline by the cloud it runs in | 16:20 |
| gryf | clarkb, not sure what limestone is :) | 16:21 |
| clarkb | it is one of the cloud providers that provides the test node resources. You can get that info by checking the zuul inventory for the job https://zuul.opendev.org/t/openstack/build/677c550cbc3843849259c1da34532c17/log/zuul-info/inventory.yaml#27 | 16:22 |
| clarkb | and that confirms it ran in limestone | 16:22 |
| gryf | I think it's OVH. | 16:22 |
| gryf | anyway. what can I do with it? should I wait a while? | 16:24 |
| clarkb | gryf: its not ovh it says in there provider: limestone-regionone | 16:25 |
| clarkb | the issue has been corrected but jobs that failed will need to be retried | 16:25 |
| gryf | clarkb, great, thank you. | 16:26 |
| *** jamesmcarthur has quit IRC | 16:32 | |
| *** jamesmcarthur has joined #openstack-infra | 16:34 | |
| *** slaweq has quit IRC | 16:34 | |
| *** slaweq has joined #openstack-infra | 16:36 | |
| *** jamesmcarthur has quit IRC | 16:38 | |
| *** tosky has quit IRC | 16:41 | |
| *** tosky has joined #openstack-infra | 16:42 | |
| fungi | gryf: also you can see in the errors you linked it was complaining about reaching https://mirror.regionone.limestone.opendev.org/ | 16:42 |
| fungi | that's the mirror server for nodes booted in the regionone region of the limestone networks provider who is one of our resource donors | 16:43 |
| gryf | fungi, oh, cool, thanks! | 16:44 |
| *** jamesmcarthur has joined #openstack-infra | 16:50 | |
| *** eolivare has quit IRC | 16:52 | |
| *** lucasagomes has quit IRC | 16:53 | |
| *** jcapitao has quit IRC | 16:59 | |
| *** gyee has joined #openstack-infra | 17:00 | |
| *** piotrowskim has quit IRC | 17:04 | |
| *** jpena is now known as jpena|off | 17:05 | |
| *** ociuhandu_ has quit IRC | 17:06 | |
| *** ociuhandu has joined #openstack-infra | 17:09 | |
| *** ociuhandu_ has joined #openstack-infra | 17:13 | |
| *** ociuhandu has quit IRC | 17:16 | |
| *** ociuhandu_ has quit IRC | 17:17 | |
| *** ociuhandu has joined #openstack-infra | 17:22 | |
| *** ociuhandu has quit IRC | 17:27 | |
| *** jamesmcarthur has quit IRC | 17:29 | |
| *** jamesmcarthur has joined #openstack-infra | 17:30 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 17:30 | |
| *** hamalq has joined #openstack-infra | 18:03 | |
| *** rpittau is now known as rpittau|afk | 18:05 | |
| *** derekh has quit IRC | 18:07 | |
| *** cloudnull has quit IRC | 18:12 | |
| *** cloudnull has joined #openstack-infra | 18:12 | |
| *** andrewbonney has quit IRC | 18:12 | |
| *** ralonsoh has quit IRC | 18:20 | |
| *** dtantsur is now known as dtantsur|afk | 18:43 | |
| *** gfidente is now known as gfidente|afk | 18:52 | |
| *** rlandy is now known as rlandy|biab | 19:04 | |
| *** jamesmcarthur has quit IRC | 19:06 | |
| *** jamesmcarthur_ has joined #openstack-infra | 19:11 | |
| *** hamalq has quit IRC | 19:29 | |
| *** paladox has quit IRC | 19:34 | |
| *** hasharAway has quit IRC | 19:36 | |
| *** paladox has joined #openstack-infra | 19:39 | |
| *** jamesmcarthur_ has quit IRC | 19:54 | |
| *** jamesmcarthur has joined #openstack-infra | 19:54 | |
| *** rlandy|biab is now known as rlandy | 20:01 | |
| *** sboyron has quit IRC | 20:09 | |
| *** jamesmcarthur has quit IRC | 20:14 | |
| *** nightmare_unreal has quit IRC | 20:16 | |
| *** slaweq has quit IRC | 20:41 | |
| *** vishalmanchanda has quit IRC | 21:41 | |
| *** rcernin has joined #openstack-infra | 21:59 | |
| *** xek has quit IRC | 22:00 | |
| *** xek has joined #openstack-infra | 22:01 | |
| *** xek has quit IRC | 22:05 | |
| melwitt | anyone seen this before? looks to be only happening on limestoneCould not connect to mirror.regionone.limestone.opendev.org:443 (216.245.200.130). - connect (113: No route to host) Could not connect to mirror.regionone.limestone.opendev.org:443 (2607:ff68:100:54:f816:3eff:feb5:4635). - connect (113: No route to host) | 22:38 |
| fungi | melwitt: when did you see it happen? the mirror server instance there had crashed or gone offline for a few hours but should have been okay by ~13:00 utc | 22:47 |
| melwitt | ah 10:44 utc | 22:47 |
| fungi | if it was before 13:00 utc today then that's probably the reason | 22:47 |
| melwitt | thank you | 22:47 |
| melwitt | I shan't bother opening any bug | 22:47 |
| fungi | melwitt: hints can be found here too: https://wiki.openstack.org/wiki/Infrastructure_Status | 22:48 |
| melwitt | oh nice, thanks. /me bookmarks | 22:49 |
| fungi | basically any time an opendev sysadmin does a "#status (log|notice|alert|ok) <message>" it gets recorded there (so long as statusbot and the wiki are both up anyway) | 22:54 |
| fungi | we try to remember to at least #status log any manual actions we take to recover things | 22:55 |
| fungi | though if it's bad enough we'll #status notice so that it gets echoed to subscribed irc channels, and if something is really bad and down we use #status alert to also update everyone's irc channel topics until we issue a #status ok | 22:56 |
| melwitt | got it. cool. I have seen the irc messages before but didn't know where the other #status things go | 22:57 |
| rm_work | anyone here familiar with gophercloud + openstack? would love to snipe someone for just a minute to ask a couple questions about passing mTLS data with connections | 23:30 |
| rm_work | IIRC I've heard some of ya'll mention gophercloud in passing :) | 23:30 |
| *** yamamoto has joined #openstack-infra | 23:32 | |
| *** dchen has joined #openstack-infra | 23:34 | |
| clarkb | I've never used it | 23:34 |
| fungi | rm_work: grepping the channel logs indicates it was mentioned in here last by TheJulia inquiring about the possibility of third-party gophercloud testing for ironic | 23:34 |
| fungi | two years ago | 23:35 |
| rm_work | lol | 23:35 |
| rm_work | yeah I was thinking more of folks at conferences saying it in person, and I know you're all here :) | 23:35 |
| *** rlandy has quit IRC | 23:36 | |
| rm_work | can't remember specifically who it was or I'd just PM T_T | 23:36 |
| fungi | rm_work: prior to that i see we spoke in a 2016 discussion about "oaktree" | 23:37 |
| rm_work | yeah maybe someone in the SDKs channel would know (though I don't think we officially support gophercloud) | 23:37 |
| fungi | right, i suspect the discussions were in the context of openstacksdk and shade | 23:38 |
| rm_work | will move my query over there, thanks | 23:39 |
| fungi | any time! | 23:39 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!