Thursday, 2017-12-07

clarkbShrews: is there a reason unlockNodeRequest() shoudl error if there is no lock?00:04
clarkbseems like that should noop instead00:04
clarkb(that would fix this problem, but I'm not sure I'm missing something)00:04
Shrewsclarkb: there will likely be more noderequest locks than noderequests b/c they are not part of the same znode (just the way we designed it).00:20
Shrewsclarkb: i'm not sure about the race. that doesn't seem likely, but too late in my day to look. will look tomorrow00:20
Shrewsclarkb: and yes, i want unlockNodeRequest to error if there is no lock b/c that really shouldn't happen (unless there's a bug)00:21
clarkbShrews: I'm also struggling with how getNodeRequestLock() returns a different type than request.lock00:21
clarkbI'm going to work on collapsing those into one type00:21
clarkbShrews: but ya reading more the more confident I am that its two conflicting/racing cleanups causing this problem.00:23
Shrewsclarkb: those are different types b/c they are fundamentally different. request locks are separate from the actual requests (described in http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html).00:28
clarkbyup I know00:28
Shrewsclarkb: don't have time to go into now though00:28
clarkbproblem is with the terminology in the code00:28
clarkbgetRequestNodeLock should requtn the node request lock00:29
clarkbits just confusing otherwise00:29
clarkbI think a single object could be both the lock content and the lock itself in kazoo/zk if we want00:29
clarkbor we can just rename things to make it a bit more clear these are not the same data00:29
*** JasonCL has joined #zuul00:55
*** JasonCL has quit IRC00:59
openstackgerritClark Boylan proposed openstack-infra/nodepool feature/zuulv3: Clarify terminology around node request locks  https://review.openstack.org/52623301:01
openstackgerritClark Boylan proposed openstack-infra/nodepool feature/zuulv3: Handle race between handler and request cleanup  https://review.openstack.org/52623401:01
clarkbjeblair: Shrews ^ thats the end result of my digging around this problem01:01
clarkbI think the chagnes in the first change should make it easier to understand when a lock is a lock or a set of znode stats or an id01:02
*** JasonCL has joined #zuul01:04
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477301:37
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image  https://review.openstack.org/51692001:37
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462001:37
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool  https://review.openstack.org/46862401:37
tristanCjeblair: thanks, here is a new ps for the static driver ^01:37
tristanCjeblair: i'll now investigate how to make the request handler generic accross openstack and static driver01:38
tristanCjeblair: mordred: and i'm still waiting for guidance regarding making the static node request using long live znode, (instead of creating new znode for each request)01:38
tristanCand thanks for all the review and +3, feels good to remove patches from staging :-)01:43
*** harlowja has quit IRC02:22
*** threestrands has joined #zuul03:52
*** threestrands has quit IRC03:52
*** threestrands has joined #zuul03:52
*** harlowja has joined #zuul04:39
*** tushar has quit IRC05:21
*** tushar has joined #zuul05:21
*** harlowja has quit IRC05:30
*** threestrands has quit IRC06:32
*** jaianshu has joined #zuul08:05
jaianshuHi...i see my instance in nodepool list gets stuck in building state, while i can see in my provider that it is active from 'nova list'. Also it continously deletes the instance and re-creates.08:07
kklimondajaianshu: check nodepool logs, it's possible that nodepool can't verify the connectivity to the machine (by trying to fetch its ssh host key) and deletes it08:12
jaianshukklimonda: so do we need to create a new dib-element for generating key in my dib image?08:18
kklimondajaianshu: well, your image should have openssh server installed and running - the keys will be generated during boot one way or another08:21
jaianshukklimonda: then how nodepool will use these keys to validate login?08:22
kklimondajaianshu: nodepool fetches ssh *host* key which is then stored in zookeeper and used by zuul to verify that it's connecting to the correct node08:23
jaianshukklimonda: where in nodepool do we need to define this ssh keys?08:24
kklimondayou don't, those are generated during boot - I think you are confusing host keys (used by client to verify the authenticity of the host) and authentication keys (used by the host to authenticate client). host keys are what's generated during boot, and nodepool has to fetch them (by using ssh-keyscan) so that zuul can verify that it's connecting to the correct machine. You can put additional ssh public keys in your image with elements if08:30
kklimondathat's what you need, but this is not what your problem is about - iirc nodepool does not try to login to the machine by itself. Anyway, you have to check logs and see why is nodepool retrying the node.08:30
*** jpena|off is now known as jpena08:32
jaianshuok..trying it08:50
*** hashar has joined #zuul09:01
*** jaianshu has quit IRC09:19
*** sshnaidm|afk is now known as sshnaidm|rover09:30
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image  https://review.openstack.org/51692009:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477309:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462009:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool  https://review.openstack.org/46862409:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement an OpenContainer driver  https://review.openstack.org/46875309:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a Kubernetes driver  https://review.openstack.org/52135609:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a generic run_handler  https://review.openstack.org/52632509:31
*** electrofelix has joined #zuul09:36
tristanCjeblair: so, i had a hard time figuring out a good design for that run_handler() method... please find in 526325 a proposal so make it generic09:38
tristanCnote that i rebased it on top of all the other drivers to make sure it's compatible, and so far so good, my setup is still working09:38
tristanCi took this oportunity to improved the k8s driver so that it creates a thread per node request using the NodeLauncherManager, it's much faster now :)09:39
tristanCanyway, the relevant files are nodepool/driver/__init__.py and nodepool/driver/openstack/handler.py, the rests are tests related09:40
*** smyers has quit IRC10:04
*** smyers has joined #zuul10:15
*** tobiash has quit IRC10:45
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477311:01
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462011:01
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image  https://review.openstack.org/51692011:01
*** tflink has quit IRC11:11
*** jkilpatr has quit IRC11:12
*** tflink has joined #zuul11:22
*** jesusaur has quit IRC11:25
*** jesusaur has joined #zuul11:28
*** jesusaur has quit IRC11:31
*** jesusaur has joined #zuul11:33
*** jkilpatr has joined #zuul11:47
*** Wei_Liu has joined #zuul11:54
Wei_Liuhello,  on which OS is recommended for zuul-scheduler/executor to run?11:55
kklimondaWei_Liu: I think it's xenial12:06
kklimondahmm, nodepool does not rebuilt "min-ready" nodes when a new image is uploaded12:07
Wei_Liukklimonda: hi, if I want to install v3 on xenial, I need to install python3 and git checkout zuul, change branch to v3, and run "pip install .", right?12:27
Wei_Liufrom zuul user?12:27
kklimondaWei_Liu: as root, it will install to /usr/local/ and then you can start it as user zuul12:28
*** jpena is now known as jpena|lunch12:40
Wei_Liukklimonda: should I use python3.5 or 3.6?12:44
*** JasonCL has quit IRC12:54
kklimondaWei_Liu: whatever comes with xenial is fine12:54
*** JasonCL has joined #zuul12:55
Wei_Liukklimonda: pip did not worked with python 3.5, I can not use "pip install .",  should I use "python setup.py isntall"13:02
kklimondaWei_Liu: what error do you get with pip install . ? (it's probably pip3 for python3 too)13:06
Wei_Liukklimonda: yes, I installed python3-pip, it worked. thanks13:12
Wei_Liukklimonda: there is no /etc/zuul directory, I create it, what access should it be?13:18
openstackgerritRico Lin proposed openstack-infra/nodepool master: Fix getProviderManager in alien_list  https://review.openstack.org/52409213:19
kklimondaWei_Liu: you need to follow documentation from https://docs.openstack.org/infra/zuul/feature/zuulv3/index.html as well as https://github.com/openstack-infra/project-config and https://github.com/openstack-infra/puppet-zuul13:20
kklimondaWei_Liu: for scheduler you'll need basic zuul config files as well as a tenant configuration13:21
*** JasonCL has quit IRC13:21
*** JasonCL has joined #zuul13:22
Wei_Liukklimonda: yes, I found it in puppet-zuul, thanks13:23
*** tobiash has joined #zuul13:23
*** tobiash_ has joined #zuul13:26
*** tobiash has quit IRC13:28
*** tobiash_ has quit IRC13:31
*** tobiash has joined #zuul13:36
*** jkilpatr has quit IRC13:38
*** jpena|lunch is now known as jpena13:46
*** tobiash has quit IRC13:46
*** tobiash has joined #zuul13:49
*** jkilpatr has joined #zuul13:50
*** tobiash has quit IRC13:53
*** tobiash has joined #zuul13:59
*** tobiash has quit IRC14:03
*** tobiash has joined #zuul14:15
pabelangerWei_Liu: any OS with python3.5+ should work, I've doing development on fedora my self. But ya, openstack-infra is running xenial14:17
*** tobiash has quit IRC14:19
*** tobiash has joined #zuul14:24
*** tobiash has quit IRC14:30
*** dkranz has joined #zuul14:34
*** tobiash has joined #zuul14:35
*** jesusaur has quit IRC14:48
*** electrofelix has quit IRC14:48
*** openstackgerrit has quit IRC14:48
*** sshnaidm|rover has quit IRC14:48
*** leifmadsen has quit IRC14:48
*** mwhahaha has quit IRC14:48
*** fungi has quit IRC14:48
*** TheJulia has quit IRC14:48
*** robcresswell has quit IRC14:48
*** fdegir has quit IRC14:48
*** logan- has quit IRC14:48
*** ChanServ has quit IRC14:48
*** qwc has quit IRC14:48
*** hashar has quit IRC14:48
*** jtanner has quit IRC14:48
*** toabctl has quit IRC14:48
*** jlk has quit IRC14:48
*** jpena has quit IRC14:48
*** nguyentrihai has quit IRC14:48
*** nhicher has quit IRC14:48
*** ianw has quit IRC14:48
*** zigo has quit IRC14:48
*** jhesketh has quit IRC14:48
*** JasonCL has quit IRC14:48
*** tushar has quit IRC14:48
*** gundalow has quit IRC14:48
*** tristanC has quit IRC14:48
*** fbouliane has quit IRC14:48
*** persia has quit IRC14:48
*** gothicmindfood has quit IRC14:48
*** odyssey4me has quit IRC14:48
*** tobiash has quit IRC14:48
*** jkilpatr has quit IRC14:48
*** _ari_ has quit IRC14:48
*** myoung|ruck has quit IRC14:48
*** rcarrillocruz has quit IRC14:48
*** lennyb has quit IRC14:48
*** EmilienM has quit IRC14:48
*** SpamapS has quit IRC14:48
*** eventingmonkey has quit IRC14:48
*** pabelanger has quit IRC14:48
*** bramwelt has quit IRC14:48
*** mwhahaha has joined #zuul14:54
*** fungi has joined #zuul14:54
*** TheJulia has joined #zuul14:54
*** robcresswell has joined #zuul14:54
*** fdegir has joined #zuul14:54
*** logan- has joined #zuul14:54
*** hashar has joined #zuul14:54
*** jtanner has joined #zuul14:54
*** qwc has joined #zuul14:54
*** leifmadsen has joined #zuul14:54
*** sshnaidm|rover has joined #zuul14:54
*** openstackgerrit has joined #zuul14:54
*** electrofelix has joined #zuul14:54
*** jesusaur has joined #zuul14:54
*** tobiash has joined #zuul14:54
*** jkilpatr has joined #zuul14:54
*** JasonCL has joined #zuul14:54
*** tushar has joined #zuul14:54
*** nguyentrihai has joined #zuul14:54
*** _ari_ has joined #zuul14:54
*** gundalow has joined #zuul14:54
*** tristanC has joined #zuul14:54
*** nhicher has joined #zuul14:54
*** jpena has joined #zuul14:54
*** myoung|ruck has joined #zuul14:54
*** ianw has joined #zuul14:54
*** toabctl has joined #zuul14:54
*** zigo has joined #zuul14:54
*** EmilienM has joined #zuul14:54
*** bramwelt has joined #zuul14:54
*** jlk has joined #zuul14:54
*** rcarrillocruz has joined #zuul14:54
*** jhesketh has joined #zuul14:54
*** lennyb has joined #zuul14:54
*** fbouliane has joined #zuul14:54
*** SpamapS has joined #zuul14:54
*** persia has joined #zuul14:54
*** eventingmonkey has joined #zuul14:54
*** pabelanger has joined #zuul14:54
*** gothicmindfood has joined #zuul14:54
*** odyssey4me has joined #zuul14:54
*** jkilpatr has quit IRC14:55
*** jkilpatr has joined #zuul14:56
*** tobiash has quit IRC14:56
*** tobiash has joined #zuul14:58
*** tobiash has quit IRC15:06
ShrewstristanC: I saw your ML post about refactoring the request handler. I don't really want to context switch away from what I'm doing right now (takes me too much time to get back on task), but I do plan to look at what you have at some point15:07
*** tobiash has joined #zuul15:07
*** tobiash has quit IRC15:11
*** tobiash has joined #zuul15:12
*** ChanServ has joined #zuul15:16
*** barjavel.freenode.net sets mode: +o ChanServ15:16
*** myoung|ruck is now known as myoung|rover15:16
jeblairkklimonda: i left a couple comments on 51516915:35
jeblairi +2d a lot of zuul changes yesterday, if another core wants to take a look at them, that would be great.  :)15:36
*** sshnaidm|rover is now known as sshnaidm|ruck15:46
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Correctly stream the remaining buffer  https://review.openstack.org/51547015:51
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add finger gateway  https://review.openstack.org/52527616:17
kklimondajeblair: thanks, I'll take a look at them tomorrow16:18
openstackgerritMerged openstack-infra/zuul-jobs master: revoke-sudo: only revoke when zuul is sudoer  https://review.openstack.org/52226116:33
openstackgerritMerged openstack-infra/zuul-jobs master: bindep: fix install bindep_command fact name  https://review.openstack.org/52550316:33
openstackgerritMerged openstack-infra/zuul-jobs master: bindep: skip package install when not needed  https://review.openstack.org/52550616:33
openstackgerritMerged openstack-infra/zuul-jobs master: Fix npm-run-test  https://review.openstack.org/51887916:33
*** dmsimard is now known as dmsimard|afk16:34
*** hashar is now known as hasharAway16:35
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add connection-port to provider diskimage  https://review.openstack.org/50411216:52
pabelangertobiash: wanted to make sure you seen comment from jeblair on https://review.openstack.org/50316617:00
tobiashpabelanger: ah thanks, overlooked that17:01
tobiashwill fix that17:01
pabelangertobiash: also a quick question on 50411217:02
pabelangerbut looks good in general17:02
tobiashpabelanger: good point, that was there in the beginning where the port was optional17:04
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add connection-port to provider diskimage  https://review.openstack.org/50411217:05
tobiashpabelanger: fixed that ^^17:05
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: WIP: Git driver  https://review.openstack.org/52561417:07
*** harlowja has joined #zuul17:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add command socket support to zuul-merger  https://review.openstack.org/52319717:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add command_socket setting to executor section  https://review.openstack.org/52346517:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Move send_command() into ZuulApp  https://review.openstack.org/52321117:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add command socket support to zuul-scheduler  https://review.openstack.org/52346617:08
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Don't gather host keys for non ssh connections  https://review.openstack.org/50316617:09
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add connection-port to provider diskimage  https://review.openstack.org/50411217:09
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Rename ssh_port to connection_port  https://review.openstack.org/50080017:10
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Support username also for unmanaged cloud images  https://review.openstack.org/50080817:12
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Add connection-type to provider diskimage  https://review.openstack.org/50314817:12
pabelangertobiash: cool, will look in a min17:13
tobiashpabelanger, jeblair: do you have the parent-change-enqueued trigger in the gate?17:15
tobiashlooks like 500799 didn't enter the gate but the depends-on change entered the gate and merged17:15
pabelangertobiash: him, is zuul and nodepool in the same shared change queue? I didn't see on status.o.o/zuul17:18
tobiashhm, good point17:21
tobiashthen we would also need parent-change-merged to support non shared queues17:21
tobiashnot sure if that's already existing17:22
SpamapSPlaying with tenants today.. as I start, I'm wondering.. is it possible I can use one repo in two tenants? I want to do that so that I can have a tenant with "deploy" pipelines and a tenant with "test" pipelines.. and I don't want them to share config.17:24
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Rename ssh_port to connection_port  https://review.openstack.org/50079917:32
jeblairtobiash: i don't think we have that trigger configured atm17:33
jeblairSpamapS: yes, you can add the same project to multiple tenants, and you can configure which types of config objects get loaded from that project independently for each tenant17:34
SpamapSjeblair: I'll let you know how that goes. Exciting times. :)17:38
jeblair++17:39
SpamapSIf it works out, I'll have a setup where Zuul deploys stuff in the gate to a stage environment and reports back on the diff it would enact in prod. Then another branch's gate will deploy to prod.17:40
SpamapSWith failed deploys in both cases simply pausing and saying "login and fix the deploy or run X to cancel and fail the gate"17:41
*** JasonCL has quit IRC17:42
*** JasonCL has joined #zuul17:45
SpamapS(and I want those deploys to be in a diff tenant than testing so that I can use super-secret-secrets and not worry about them being abused)17:46
*** jpena is now known as jpena|off17:48
*** JasonCL has quit IRC17:52
*** JasonCL has joined #zuul17:53
jeblairSpamapS: cool, i'd like to hear how that goes.  once you get that going, i wonder if, with some combination of final and protected jobs and config repos, you could accomplish that in one tenant.17:54
*** Wei_Liu has quit IRC17:55
SpamapSjeblair: that's a good question, I wondered that too.17:57
SpamapSjeblair: for now one problem I have is a lot of secrets just live in private repos... I don't quite have time to convert it all to zuul secrets.17:58
jeblairSpamapS: yeah, i think you're on the right track with that caveat; refactor later.  :)18:03
tobiashjeblair: is 525239 ready to land or is some cleanup left in openstack world?18:09
jeblairtobiash: there may be a few changes left that haven't merged, but they've had plenty of time.  i'm ready to land it.  i'll go ahead and +w it so i take the blame.  :)18:19
tobiash:)18:21
*** vivsoni__ has joined #zuul18:24
*** vivsoni_ has quit IRC18:24
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add support for protected jobs  https://review.openstack.org/52298518:25
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove implied playbook extensions  https://review.openstack.org/52523918:36
*** sshnaidm|ruck is now known as sshnaidm|off18:43
*** jkilpatr has quit IRC18:52
*** electrofelix has quit IRC19:08
*** jkilpatr has joined #zuul19:13
*** jkilpatr has quit IRC19:18
*** jkilpatr has joined #zuul19:19
*** harlowja has quit IRC19:21
*** ricky_vaca has joined #zuul19:31
*** JasonCL has quit IRC19:33
*** JasonCL has joined #zuul19:35
*** Wei_Liu has joined #zuul19:41
*** JasonCL has quit IRC19:59
ricky_vacam20:03
*** JasonCL has joined #zuul20:03
*** jkilpatr has quit IRC20:08
*** jkilpatr has joined #zuul20:24
*** hasharAway is now known as hashar20:31
*** jkilpatr has quit IRC20:32
*** dmellado has quit IRC20:37
*** dmellado has joined #zuul20:50
*** jkilpatr has joined #zuul21:06
*** nguyentrihai has quit IRC21:29
*** dmsimard|afk is now known as dmsimard21:30
*** hashar has quit IRC21:43
*** jtanner has quit IRC21:43
*** hashar has joined #zuul21:48
*** jtanner has joined #zuul21:48
*** openstackgerrit has quit IRC21:48
*** jkilpatr has quit IRC21:53
*** tobiash has quit IRC22:01
*** tobiash has joined #zuul22:01
*** threestrands has joined #zuul22:04
*** hashar has quit IRC22:14
*** openstackgerrit has joined #zuul22:23
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add finger gateway  https://review.openstack.org/52527622:23
*** weshay is now known as weshay|ruck22:23
Shrewswth... how is that in merge conflict22:24
Shrewsugh22:24
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add finger gateway  https://review.openstack.org/52527622:27
*** jtanner has quit IRC22:36
*** leifmadsen has quit IRC22:39
*** jtanner has joined #zuul22:42
*** leifmadsen has joined #zuul22:42
*** leifmadsen has quit IRC22:42
*** leifmadsen has joined #zuul22:42
*** harlowja has joined #zuul23:07
*** dkranz has quit IRC23:14
*** jkilpatr has joined #zuul23:40
*** threestrands has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!