Thursday, 2017-03-16

*** yuanying has quit IRC00:10
*** pmannidi has quit IRC00:11
*** pmannidi has joined #openstack-kuryr00:11
*** yuanying has joined #openstack-kuryr00:15
*** hongbin has quit IRC00:17
*** yedongcan has joined #openstack-kuryr00:52
*** limao has joined #openstack-kuryr00:57
*** hongbin has joined #openstack-kuryr01:53
*** yuanying has quit IRC03:06
*** leifmadsen has quit IRC03:26
*** russellb has quit IRC03:26
*** leifmadsen has joined #openstack-kuryr03:27
*** russellb has joined #openstack-kuryr03:27
*** vikasc has quit IRC04:12
*** aojea has joined #openstack-kuryr04:18
*** aojea has quit IRC04:24
*** hongbin has quit IRC04:30
*** yuanying has joined #openstack-kuryr04:46
*** tonanhngo has joined #openstack-kuryr05:04
*** tonanhngo has quit IRC05:22
*** saneax-_-|AFK is now known as saneax05:34
openstackgerritMerged openstack/kuryr-kubernetes master: Adding vagrant configs to kuryr-kubernetes  https://review.openstack.org/44516205:41
*** janki has joined #openstack-kuryr05:54
*** vikasc has joined #openstack-kuryr05:56
*** aojea has joined #openstack-kuryr06:20
*** aojea has quit IRC06:25
*** aojea has joined #openstack-kuryr06:48
*** aojea has quit IRC06:53
*** aojea has joined #openstack-kuryr06:58
openstackgerritMerged openstack/kuryr-kubernetes master: devstack: fix run_kubelet data directory  https://review.openstack.org/44279407:07
openstackgerritMerged openstack/kuryr-kubernetes master: devstack: Use stackrc DATA_DIR by default  https://review.openstack.org/44284607:15
*** pmannidi has quit IRC07:29
*** danil has joined #openstack-kuryr07:42
*** aojea has quit IRC08:06
*** ltomasbo|away is now known as ltomasbo08:18
*** yuanying has quit IRC08:19
*** pcaruana has joined #openstack-kuryr08:30
irenabltomasbo, hi08:33
*** aojea has joined #openstack-kuryr08:33
irenabltomasbo, I went throught your serias of patches, agree with ivc_ that we need to agree mfirst on the devref08:34
ltomasbohi irenab, yes, we need to agree on devref08:34
ltomasbosince I used to have just one driver as ivc_ suggest, and move to a more separate implementation based on his comments08:37
irenabltomasbo, please check my comments regarding delagation instead of inheritance08:41
*** yuanying has joined #openstack-kuryr08:55
ltomasboirenab, I'll do09:05
ltomasboirenab, thanks!09:05
*** aojea has quit IRC09:16
*** aojea has joined #openstack-kuryr09:17
*** aojea has quit IRC09:17
*** aojea has joined #openstack-kuryr09:18
*** limao has quit IRC09:25
dmelladomorning folks09:25
dmelladoapuimedo: ping I wasn't xD09:25
apuimedodmellado: -109:25
apuimedo:-)09:25
dmelladolol09:25
apuimedoI forgot what I wanted09:25
apuimedoso you get a free -1 on your next patch :P09:25
dmelladoI'll reply with a random -1 on some on your patches09:27
dmelladoand you wouldn't even know why09:27
dmelladoIn fact I'll write a 'punishment script'09:27
dmelladowhich will also add jeff as a reviewer09:27
apuimedo:D09:28
dmelladoirenab: do you think that it would be punishment enough? :D09:28
apuimedoyou are evil09:28
dmelladoI know xD09:29
*** garyloug has joined #openstack-kuryr09:45
*** aojea has quit IRC09:53
*** ltomasbo is now known as ltomasbo|away10:00
*** ltomasbo|away is now known as ltomasbo10:11
ltomasbohi ivc_, I've tried the new version of the patches (for baremetal) and now I'm getting what you commented yesterday10:13
ltomasboI believe the reason is I was getting some warning before the lbaas support patch got merged yesterday10:14
ltomasboand perhaps that was making the pool driver to be executed10:15
ltomasboI did not hit that problem before for dozens of runs, but now I'm hitting it10:15
*** aojea has joined #openstack-kuryr10:22
*** aojea has quit IRC10:22
*** aojea has joined #openstack-kuryr10:24
apuimedoivc_: ping10:28
irenabdmellado, I prefer empowerment over punishment :-)10:59
*** yedongcan has quit IRC10:59
*** yedongcan has joined #openstack-kuryr11:00
*** yedongcan has left #openstack-kuryr11:00
ltomasbohi irenab11:11
*** aojea has quit IRC11:11
irenabltomasbo, hi11:11
ltomasboI just took a look at your comment here: https://review.openstack.org/#/c/436877/9/kuryr_kubernetes/controller/drivers/generic_pool_vif.py11:11
ltomasboabout delegation vs inheritance11:11
ltomasboNot sure I got why you prefer delegation here11:12
ltomasbothe genericpoolpodVIFDriver is actuall the PodVifDriver, not the pool one (as in the actual patch)11:12
ltomasboso, those are the rquest_vif and release_vif functials directly called by the VIF handler11:13
ltomasboSo, at the only functions changed from the already existing generic_vif driver are the request and release, I went for inheritance11:14
ltomasboto not have to explicitly include the rest of the methods being use11:14
openstackgerritAntoni Segura Puimedon proposed openstack/kuryr-kubernetes master: VIF Pool  https://review.openstack.org/44646211:14
*** aojea has joined #openstack-kuryr11:15
ltomasboso, probably I'm missing something there from your comment11:15
apuimedohttps://review.openstack.org/#/c/446462/11:16
apuimedoltomasbo: vikasc: irenab: ivc_: ^^11:16
apuimedoThis patch is just for discussion11:16
vikascthanks apuimedo.11:17
irenabltomasbo, my point was that PodVifDriver can call port_allocator and this will either create port or get from the pool11:20
dmelladoirenab: :D11:22
ltomasboumm11:22
ltomasboirenab, that for the delegation/inheritance discussion?11:22
irenabltomasbo, yes, so no need to have new poolVifDriver11:23
ltomasboor for the discussion about the devref and follow up on the discussion with ivc_ and the patch set apuimedo just pasted?11:23
irenabltomasbo, I didn't check  the patch yet, but we had some discussion previously11:24
irenabltomasbo, I suggest not inherit at all, and modify existing vif driver to call PortAllocatorAPI (this is the delegation I was talking about)11:26
ltomasboapuimedo, so, the pooldriver will have the vif_driver instance included11:26
ltomasboand the vif handler using this pooldriver11:26
ltomasboinstead of the current vifdriver11:26
ltomasboI see that idea somehow similar to what I proposed initially, but keeping the separation of the drivers11:28
*** aojea has quit IRC11:28
ltomasbomy first approach was to have an extended driver based on the generic vif driver that fallback to the generic one when there were no ports available11:29
ltomasboapuimedo, similar to what you have there11:29
ltomasbobut without using the delegation as irenab propose11:29
*** aojea has joined #openstack-kuryr11:29
ltomasboI like this approach and feel it will be easier to read and maintain11:30
irenabltomasbo, yes, just going through the patch and trying to comment to reflect possible modification11:30
irenabI think pool is more of internal implementation for getting port, so it should not be at Handler level11:31
ltomasboirenab, apuimedo, I agree with this11:35
irenabapuimedo, posted the comment about the difference, but in general had similar idea. Its just the matter of the what comes first, PodVifDriver -> PoolDriver or PoolDriver->PodVifDriver11:35
ltomasbomy main concern is how to populate the pools, not during the pod creation11:36
ltomasboperhaps, one options is to modify a bit what apuimedo just proposed11:37
ltomasboand in the except IndexError11:37
ltomasboperhaps we can just do a call to request_vif to get one port, for the pod11:38
ltomasboand do the population right after that11:38
apuimedoI have to go to the doctor. I'll read up when I return11:46
irenabltomasbo, do you want to populate out of scope of the Pod creation operation?11:46
ltomasboirenab, yes11:46
*** aojea has quit IRC11:46
ltomasboI don't want the pod to be waiting for the creation of 10 ports11:47
ltomasboor 20 or whatever number11:47
ltomasbosince that will lead to some containers needed a few seconds to boot (1-15)11:47
ltomasbowhile other could take minutes11:47
*** gsagie has joined #openstack-kuryr11:48
ltomasboalso, I want to 're-fill' the pool11:48
ltomasbonot when it is empty, but when it is below a certain amount11:48
irenabactually, with 3rd party resource it can be used to set a request and then get event to create ports11:51
openstackgerritMerged openstack/kuryr master: [Fix gate]Update test requirement  https://review.openstack.org/44024711:58
openstackgerritMerged openstack/kuryr-kubernetes master: [Fix gate]Update test requirement  https://review.openstack.org/44024812:14
openstackgerritOpenStack Proposal Bot proposed openstack/kuryr master: Updated from global requirements  https://review.openstack.org/43860412:28
openstackgerritOpenStack Proposal Bot proposed openstack/kuryr-libnetwork master: Updated from global requirements  https://review.openstack.org/43196612:28
openstackgerritOpenStack Proposal Bot proposed openstack/kuryr-kubernetes master: Updated from global requirements  https://review.openstack.org/44649212:28
*** garyloug_ has joined #openstack-kuryr13:35
*** garyloug has quit IRC13:37
*** janki has quit IRC13:39
*** vikasc has quit IRC13:43
*** danil has quit IRC14:02
apuimedoltomasbo: ok. I've read up on it14:03
apuimedoI'd like to hear what ivc_ makes out of it too14:03
ltomasbohi apuimedo14:04
apuimedohi14:04
ltomasbodo you know if there has been a change to kuryr-kubernetes14:04
ltomasbodevstack related?14:04
ltomasbowe are trying to deploy kuryr-kuberentes with devstack and the kubelet gets not started14:05
ltomasboand if I try to execute kubelet with the same command (with the proper ip) from the other devstack (deployed a few days ago)14:07
*** gsagie has quit IRC14:07
ltomasbothe kubelet starts, but I cannot create containers as it says14:09
ltomasboFailedToDropPrivileges: privsep helper command exited non-zero (1)14:09
ltomasbohelpers.go:101] Unable to get network stats from pid 8967: couldn't read network stats: failure opening /proc/8967/net/dev: open /proc/8967/net/dev: no such file or directory14:09
ltomasboE0316 16:07:40.083948    5638 docker_manager.go:357] NetworkPlugin cni failed on the status hook for pod 'busybox-sleep1' - Unexpected command output Device "eth0" does not exist.14:09
ltomasbodo you know a possible reason? it was working like a charm before...14:10
*** vikasc has joined #openstack-kuryr14:14
apuimedoltomasbo: is it a new deployment based on centos7?14:15
ltomasboyes14:15
ltomasbocentos 7.314:15
*** saneax is now known as saneax-_-|AFK14:16
apuimedosomebody reported it14:16
apuimedoand running kubelet by hand fixed it14:16
dmelladoheh, so *yes*14:16
dmelladoit is14:16
dmelladorunning it by hand didn't really14:16
dmelladojust crashed in some weird way14:16
ltomasbothat is a new failure then14:16
ltomasboI tried with centos7 a couple of days ago14:16
apuimedoltomasbo: selinux?14:16
ltomasbos/a couple of days/a couple of weeks14:16
dmelladoltomasbo: should we put it to permissive, just in case?14:16
apuimedotry14:16
apuimedoit's always the prime suspect14:18
dmelladoapuimedo: I thought the prime suspect was always Gabert14:18
dmelladowith chorizuru14:18
*** vikasc has quit IRC14:18
apuimedono, it's selinux, then firewalld14:18
ltomasboit is not working either14:21
dmelladoapuimedo: we're going to give you a nice message14:21
ltomasbohttp://paste.openstack.org/show/602973/14:21
ltomasbonot nice, it is not working...14:22
dmelladothere you go \o/14:22
ltomasboapuimedo, do you know if the bug comes from a modification in kuryr-kuberentes lately or it is related to some changes at devstack?14:23
apuimedoltomasbo: looks to me like a breakage in os-vif14:24
*** pcaruana has quit IRC14:24
apuimedowith the privsep usage14:24
ltomasbonot familiar with privsep...14:25
ltomasboany easy way to avoid this?14:26
dmelladomaybe use stable devstack?14:26
dmellado'stable'14:26
apuimedoltomasbo: try to downgrade os-vif14:29
ltomasboapuimedo, I got it working with a devstack form 17th february14:33
ltomasboand with the same os-vif (1.4.0) that I have now14:33
ltomasbogoing to try reverting devstack...14:33
apuimedoltomasbo: compare pip lists14:34
dmelladoapuimedo: is there any bug opened for that centos7 stuff?14:34
apuimedodmellado: nope14:34
dmellado'awesome'14:35
apuimedodmellado: I see you volunteer to file it14:35
dmelladoLOL14:35
dmelladowe'll get to open it after checking14:35
ltomasboos-vif (1.4.0)14:35
ltomasboin both of them14:35
ltomasbothe weird thing to me is that the kubelet screen is empty14:36
ltomasbois not that the command fails, is that there is simply no command14:37
apuimedoltomasbo: and oslo.privsep?14:37
ltomasbothen, If I run it manually, it works, but the container get stuck with those errors14:37
ltomasbolet me check14:37
ltomasboumm14:38
ltomasbothere is a difference there14:38
ltomasbo0.17 not working14:38
ltomasbo0.16 working14:38
apuimedo:-)14:38
apuimedotry to replace 0.17 in the not working scenario14:39
apuimedomaybe it's just that14:39
*** hongbin has joined #openstack-kuryr14:40
ltomasbona, not working either14:41
ltomasbosame error14:41
ltomasbo Warning MissingClusterDNS       kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy14:42
apuimedoltomasbo: make a full diff of the deps14:42
apuimedopip14:42
*** vikasc has joined #openstack-kuryr14:46
ltomasbodocker-py and docker-pycreds14:53
ltomasboare not there in the new one14:53
dmelladoapuimedo: let's say that *I* get you a fuet if you figure out all the pip diffs xD14:54
ltomasboapuimedo, there are a lot of differences...14:54
apuimedoltomasbo: sort them and run them through diff!14:56
apuimedo:-)14:56
apuimedochecking one by one is a lot of work14:56
apuimedodmellado: I don't think you can top my fuet14:56
dmelladoyou'd never know!14:56
dmelladowe can get you some creds and then wonder about the fuets14:56
dmelladoxD14:56
ltomasbowe are using vimdiff14:56
ltomasboso, it is sorted and compared14:57
ltomasbostill 293 differences...14:57
ltomasbopyroute2 0.4.12 a 0.4.1314:58
ltomasbos/a/to14:58
apuimedoltomasbo: 0.4.13 brought some biggish change iirc15:02
apuimedocan you try with 0.4.12?15:02
ltomasbogoing to, that is a good candidate15:03
ltomasbosame error15:08
ltomasbo  File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 85, in plug15:09
ltomasbo    raise os_vif.exception.PlugException(vif=vif, err=err)15:09
ltomasboPlugException: Failed to plug VIF VIFOpenVSwitch(active=True,address=fa:16:3e:e5:57:31,bridge_name='br-int',has_traffic_filtering=True,id=8b63960a-b718-456a-926d-32b41aacb083,network=Network(945b5fef-10c1-426d-8c8b-aaf4fd9f61c6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8b63960a-b7'). Got error: privsep helper command exited non-zero (1)15:09
ltomasbo; Skipping pod15:09
apuimedoltomasbo: the one working is f25 or centos7?15:10
ltomasbothe one working is also centos715:10
ltomasbo7.315:10
dmelladoapuimedo: works on f2515:10
ltomasboexacly the same version15:10
ltomasbos/exacly/exactly15:10
apuimedodmellado: ltomasbo: borked ovs on centos?15:10
ltomasbothe ovs is different15:11
ltomasboas I manually installed (in the working one) a complited version to go fomr 2.5 to 2.6.115:11
apuimedoalso, you should be able to reproduce the failure with a nova devstack15:11
apuimedoltomasbo: could you try to put 2.6.1 on the failing one?15:11
dmelladoapuimedo: what do you mean by nova devstack, disabling neutron?15:12
dmelladoovs=2.715:12
ltomasbo ovs-vsctl --version                                                    │        get_container "$KURYR_HYPERKUBE_IMAGE" "$KURYR_HYPERKUBE_VERSION"15:12
ltomasboovs-vsctl (Open vSwitch) 2.6.1                                                           │        prepare_kubernetes_files15:12
ltomasboDB Schema 7.14.015:12
ltomasbouppss, some lines get in between15:12
ltomasbobut ovs-vsctl already shows 2.6.1 (though the pip package shows 2.7.0, not sure why)15:13
apuimedodmellado: I mean starting a VM15:13
ltomasbook, let me try that15:14
ltomasboyou are right15:18
ltomasboit is stuck in build status15:18
apuimedo:-)15:18
ltomasbosame error with 2.6.1 python bindings too15:20
apuimedowhere did you get this el7.3?15:21
dmelladoapuimedo: centos715:23
dmelladolatest15:23
dmelladojust fetch a qcow2 mirror15:23
ltomasboI gave up and did another stack.sh15:23
ltomasboremoving the build from source for ovs15:23
ltomasbolet's see if that was the problem15:24
apuimedook15:27
alraddarlaapuimedo, quick question regarding https://review.openstack.org/#/c/444998/2/doc/source/specs/pike/contrail_support.rst15:27
apuimedoalraddarla: sure15:28
alraddarlaWould i need to somehow add an opencontrail gate as well? is that what is being asked in the comments15:28
apuimedoI think for starters, having a gate that just installs all the components would be fine15:33
apuimedo(apart from unit tests)15:33
alraddarlaokay cool. thank you apuimedo15:35
apuimedoyou're welcome15:36
dmellado+1 on the gate thingy15:36
dmelladooverall15:36
dmelladoapuimedo: let's have a chat/bp re: this when I'm back from bucharest15:37
apuimedodmellado: regarding what?15:38
dmelladoapuimedo: CI/gates15:38
dmelladoand fuet, of course15:39
apuimedoah15:39
openstackgerritDarla Ahlert proposed openstack/kuryr-kubernetes master: Spec for adding contrail support in kuryr-k8s  https://review.openstack.org/44499816:04
*** aojea has joined #openstack-kuryr16:07
openstackgerritDarla Ahlert proposed openstack/kuryr-kubernetes master: Spec for adding contrail support in kuryr-k8s  https://review.openstack.org/44499816:11
apuimedoltomasbo: let me know how the new stacking goes16:17
ltomasbonew stack does not create the kubelet screen either16:24
apuimedo:/16:25
apuimedothat's very weird16:25
ltomasbonot working, same errors16:27
apuimedoit was a clean centos7, right?16:27
ltomasboyes16:27
ltomasbonot the last stacks16:27
dmelladoapuimedo: yep, originally16:27
ltomasbobut the first one was a fresh installation16:27
dmelladowe'll switch to fedora, just ot check...16:28
apuimedodmellado: ltomasbo: please do16:29
ltomasbosuch a productive day... :D16:29
dmelladototally16:30
dmelladonext time I'll just bring the switch (not the network one) and show you my progress on Zelda16:30
dmelladoxD16:30
apuimedodmellado: I'm so slow at it16:30
apuimedoI'm just got back my first memory :/16:30
dmelladoapuimedo: then you're not that bad16:30
dmelladoI went around exploring16:30
dmelladoa lot16:31
dmelladountil a ton of 'strong' folks showed up and killed me xD16:31
apuimedoand just there there's a fscking invincible minotaur that throws me freezing arrows16:31
apuimedokilled in one shot16:31
dmelladoapuimedo: lol16:31
dmelladothat one killed me too16:31
dmelladowith ice arrows xD16:31
apuimedo:/16:32
*** aojea has quit IRC16:43
*** aojea_ has joined #openstack-kuryr16:48
*** ltomasbo is now known as ltomasbo|away16:50
*** aojea_ has quit IRC16:52
*** ltomasbo|away is now known as ltomasbo17:01
*** aojea has joined #openstack-kuryr17:29
*** ltomasbo is now known as ltomasbo|away18:03
*** aojea has quit IRC18:04
*** garyloug_ has quit IRC18:06
openstackgerritHongbin Lu proposed openstack/kuryr-libnetwork master: Add support for specified v6 subnetpool  https://review.openstack.org/44163818:11
openstackgerritOpenStack Proposal Bot proposed openstack/fuxi master: Updated from global requirements  https://review.openstack.org/43108518:12
*** tonanhngo has joined #openstack-kuryr18:18
*** tonanhngo has quit IRC18:22
*** tonanhngo has joined #openstack-kuryr18:25
openstackgerritHongbin Lu proposed openstack/kuryr-libnetwork master: [WIP] Process v6 pool's uuid option in create_network  https://review.openstack.org/44670619:26
*** neiljerram has quit IRC19:28
*** neiljerram has joined #openstack-kuryr20:38
*** kzaitsev_mb has joined #openstack-kuryr21:28
*** kzaitsev_mb has quit IRC21:38
openstackgerritHongbin Lu proposed openstack/kuryr-libnetwork master: Process v6 pool's uuid option in create_network  https://review.openstack.org/44670621:47
*** kzaitsev_mb has joined #openstack-kuryr22:16
openstackgerritHongbin Lu proposed openstack/kuryr-libnetwork master: Add fullstack test for containers with dual-net  https://review.openstack.org/44466122:31
*** neiljerram has quit IRC22:53
*** kzaitsev_mb has quit IRC22:54
*** pmannidi has joined #openstack-kuryr23:14
openstackgerritHongbin Lu proposed openstack/kuryr-libnetwork master: [WIP] Reset neutron existing port on release_address  https://review.openstack.org/44568323:41
*** hongbin has quit IRC23:43

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!