Thursday, 2020-05-14

*** dobson has joined #openvswitch00:11
*** vdasari has joined #openvswitch01:11
*** ihrachys has quit IRC01:12
*** vdasari has quit IRC01:19
*** vdasari has joined #openvswitch01:20
*** vdasari has quit IRC01:24
*** vdasari has joined #openvswitch01:24
*** vdasari has quit IRC01:28
*** vdasari has joined #openvswitch01:28
*** dalvarez has quit IRC01:48
*** dholler has quit IRC02:07
*** dholler has joined #openvswitch02:19
*** dalvarez has joined #openvswitch02:30
*** acidfu_ has quit IRC02:40
*** psahoo has joined #openvswitch02:50
*** vdasari has quit IRC03:06
*** zhouhan_ has quit IRC03:35
*** zhouhan_ has joined #openvswitch03:37
*** anilvenkata has joined #openvswitch04:55
*** lamawithonel has quit IRC05:07
*** mmichelson has quit IRC05:29
*** links has joined #openvswitch05:36
*** mmichelson has joined #openvswitch05:41
*** psahoo has quit IRC05:47
*** anilvenkata_ has joined #openvswitch05:53
*** psahoo has joined #openvswitch05:55
*** anilvenkata has quit IRC05:56
*** eelco has joined #openvswitch06:02
*** anilvenkata has joined #openvswitch06:05
*** anilvenkata_ has quit IRC06:06
*** psahoo has quit IRC06:33
*** psahoo has joined #openvswitch06:36
*** anilvenkata has quit IRC06:38
*** anilvenkata has joined #openvswitch06:38
*** mmirecki has joined #openvswitch06:47
*** slaweq has joined #openvswitch07:09
*** psahoo has quit IRC07:29
*** psahoo has joined #openvswitch07:37
*** dceara has joined #openvswitch07:51
*** jaicaa has quit IRC08:14
*** jaicaa has joined #openvswitch08:17
*** gmg has joined #openvswitch08:27
*** gmg has quit IRC08:55
*** timothy has joined #openvswitch09:27
*** jraju__ has joined #openvswitch10:05
*** links has quit IRC10:05
*** zhouhan_ has quit IRC10:17
*** zhouhan has joined #openvswitch10:17
numansbern, probably you can explore OVN :)10:17
bernnumans: thanks, there are so many rabbit holes for me to explore during lockdown ;)10:20
numans:)10:20
bernI hope to be able to blog something along the lines of ovs for mortals10:20
bernat (that mythical) some point10:21
bernI watched the video flaviof posted yesterday; I will need to watch it again. If I can understand enough about OVN/OVS/Faucet with the knowledge I already have about enterprise networking, I hope to invent whole new rabbit holes and hopefully even emerge with something useful.10:23
numansAll the best.10:23
bern:)10:23
*** JamesBenson has joined #openvswitch11:41
*** psahoo has quit IRC12:08
*** psahoo has joined #openvswitch12:25
*** troulouliou_div2 has joined #openvswitch12:32
*** ihrachys has joined #openvswitch12:36
*** vdasari has joined #openvswitch12:53
*** bostondriver has joined #openvswitch13:01
*** psahoo has quit IRC13:03
*** acidfu_ has joined #openvswitch13:09
*** jraju__ has quit IRC13:22
*** vdasari has quit IRC13:23
*** matteo has quit IRC14:10
*** matteo has joined #openvswitch14:11
*** mmirecki has quit IRC14:16
*** dcbw has joined #openvswitch14:19
*** gmg has joined #openvswitch14:47
*** gmg has quit IRC14:49
*** slaweq has quit IRC15:06
*** slaweq has joined #openvswitch15:11
*** eelco has quit IRC15:15
*** gmg1 has joined #openvswitch15:38
*** gregwork has joined #openvswitch16:03
*** acidfoo_ has joined #openvswitch16:16
*** acidfu_ has quit IRC16:17
*** zhouhan_ has joined #openvswitch16:51
*** dholler has quit IRC16:53
*** zhouhan has quit IRC16:55
*** timothy has quit IRC17:07
*** ryzhyk has joined #openvswitch17:14
mmichelsonHi everyone, I'm going to start the meeting17:14
*** blp has joined #openvswitch17:14
_lore_hi all17:14
blpMorning all.17:14
mmichelson#startmeeting ovn-community-development-discussion17:14
openstackMeeting started Thu May 14 17:14:56 2020 UTC and is due to finish in 60 minutes.  The chair is mmichelson. Information about MeetBot at http://wiki.debian.org/MeetBot.17:14
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:14
openstackThe meeting name has been set to 'ovn_community_development_discussion'17:15
mmichelsonJust as a reminder, we're coming up on hard freeze. Thanks to everyone for mentioning which patches they want included in 20.0617:15
mmichelsonI'm planning to spend time today and tomorrow to help with the review effort on those patches17:15
mmichelsonI plan to create the 20.06 branch on Monday17:16
mmichelsonAs far as my activity goes, After doing the GROUP_MOD message split patch, I took on some of the smaller issues that have been reported as of late. Thanks to everyone who has helped review those patches17:16
mmichelsonAnd that's all from me.17:17
blpI have a quick update.17:17
blpI rebased the OVN DDlog code against master earlier this week. All the tests pass.17:17
blpryzhyk has been working on performance.17:17
blpThat's all I have for the moment, unless there are questions.17:18
ryzhykYes, I am making progress on performance, but we need scale tests.17:18
mmichelsonblp, ryzhyk awesome!17:18
ryzhykSo far I've been using Han's old scale test log, but it is getting increasingly irrelevant with all the changes to OVN since it was created.17:19
ryzhyk(that's it from me)17:19
numansryzhyk, If you want explore this one - https://github.com/dceara/ovn-heater17:20
numansfor the scale testing.17:20
ryzhyknumans: thanks!17:20
numansBut we are planning to run a scale test with the ddlog changes. We need to figure out a way to compile ovn-northd-ddlog in the container images. Once that is done, it should be straightforward.17:21
numansI can go real quick.17:21
blpYeah, we'll do some preliminary work and then we can figure out how to do that.17:21
numansOk. sounds good.17:22
numansI did some reviews this week. And I spent much of the time refactoring/reworking on patch 1 and 2 of my I-P patch series17:22
numansThanks to dceara for the reviews.17:22
numansI'll continue to do that and planning to submit v6 by tomorrow.17:23
numansI appreciate more reviewers joining in :).17:23
numansThat's it from me.17:23
dcearahi all17:24
*** zhouhan_ has quit IRC17:25
dcearanumans, ryzhyk I can hack ovn-heater to compile ovn-northd-ddlog for the scale test container images. I just need to know what branches to use17:25
*** zhouhan has joined #openvswitch17:25
zhouhannumans: sorry that I didn't get time to review your I-P patches last week. I will resume this week.17:25
numanszhouhan, thanks.17:25
blpdceara: We're in a little bit of a transition at the moment, we'll get back to you on that.17:26
dcearablp, sure17:26
zhouhancan someone pin the link to the meeting logs?17:27
mmichelsonzhouhan, I don't understand what you mean by "pin"17:28
flaviofzhouhan: http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/17:28
numansflaviof, you forgot to use the #link :)17:28
flaviofLOL17:28
zhouhanmmichelson: I meant, pin in this IRC channel, like the "FAQ: http://docs.openvswitch.org/en/latest/faq/"17:29
flaviofthere is a handy link to that dir in ovn.org too17:29
zhouhanthanks flaviof17:29
mmichelsonah ok17:29
zhouhanMay I go next?17:30
mmichelson#topic Open vSwitch, a Linux Foundation Collaborative Project || FAQ: http://docs.openvswitch.org/en/latest/faq/ || Hyper-V meeting Tues 10:00 Pacific || OVN meeting Thurs 10:15 am US Pacific || Use ovs-discuss@openvswitch.org for questions if you don't get an answer here. || OVN weekly meeting logs can be found at: http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/17:30
mmichelsonoh crap it's /topic isn't it17:30
mmichelsonAnd I don't have permission17:31
mmichelsonzhouhan, go ahead17:31
zhouhanFirstly I have a question regarding the OVS FAQ on the compatibility17:31
blpI can change the topic.17:31
zhouhan2.11.x3.10 to 4.1817:31
zhouhan2.12.x3.10 to 5.017:31
zhouhan2.14.x3.10 to 5.517:31
*** ChanServ sets mode: +o blp17:31
zhouhanIt didn't mention 2.13, why is that?17:31
*** blp changes topic to "Open vSwitch, a Linux Foundation Collaborative Project || FAQ: http://docs.openvswitch.org/en/latest/faq/ || Hyper-V meeting Tues 10:00 Pacific || OVN meeting Thurs 10:15 am US Pacific || Use ovs-discuss@openvswitch.org for questions if you don't get an answer here. || OVN weekly meeting logs can be found at: http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/"17:31
zhouhanblp: do you know?17:31
*** ChanServ sets mode: -o blp17:32
flaviofblp++17:32
*** dcbw has quit IRC17:32
blpzhouhan: Probably just overlooked.17:32
zhouhanWe tried 2.13 compiling with 5.4, it has a message: configure: error: Linux kernel in /lib/modules/5.4.0-31.generic.x86_64/build is version 5.4.0, but version newer than 5.0.x is not supported (please refer to the FAQ for advice)17:33
zhouhanSo it seems not an overlook, but on purpose ...17:33
zhouhanIt's confusing though17:33
zhouhanI continued debugging the problem of: deferred action limit reached, drop record action17:34
blpzhouhan: It looks like 2.13.x supports the same versions as 2.12.x.17:34
zhouhanok, thanks blp17:35
zhouhanI think I made some progress on the endless recirc problem. The issue was that there is slowpath required for the actions while there is also a group action which requires dp_hash + recirc17:35
blpOK, I sent a patch to update the FAQ.17:36
blpzhouhan: Oh that's a little awkward. Do you have a lead on a fix?17:36
zhouhanWhenever this combination comes, it tries to execute dp_hash in userspace, and only do recirc in kernel17:36
zhouhanHowever, the usespace hash generated is not carried for injecting the packet to datapath, so after recirc back, the upcall doesn't have dp_hash value, again.17:37
blpOh. Have you figured out why we don't pass the dp_hash back?17:38
zhouhanSo when it hits the same group action, it generates recirc and dp_hash actions again17:38
blpIs that the easy fix?17:38
zhouhanAt this time, the recirc_id generated is the same as the older one because all the metadata in state is the same, which caused the loop17:38
zhouhanblp: I haven't got time on the fix yet.17:39
zhouhanblp: I think there are two options17:39
blpUserspace does know how to put dp_hash into a flow, see odp_flow_key_from_flow__().17:39
zhouhanthanks for the pointer!17:39
blpIt will do so if it detects recirc support in the datapath. Perhaps it's not being detected properly?17:40
blpI think we log whether that feature is detected as supported. Check the log, if that's the problem then we should fix the detection logic.17:40
zhouhanI think this is one option. The other option is always do dp_hash in datapath instead of "trying to help" in userspace17:40
blpI think there is some reason why we do that, although I don't recall what it was.17:40
zhouhanI wonder if dp_hash anyway requires recirc in datapath, why would it help by doing dp_hash in userspace17:41
* numans says bye and disappears.17:41
blpWhat's the reason that it gets slow-pathed to begin with?17:41
zhouhanThe other question is why is there slowpath required in the first place17:41
* blp waves at numans17:42
zhouhanblp: exactly17:42
blp"trace" should explain the reason for slow-pathing.17:42
zhouhanblp: the scenario is ping LRP's IP. The LR is replying ICMP by simply setting fields, and I can't tell why is slowpath needed17:43
blpSome fields aren't supported for setting in the datapath, that could be the reason.17:43
blpBut ofproto/trace should say. For example, if it's because of fields that the datapath can't set, it should say something about unsupported actions.17:44
imaximetsblp, zhouhan: execute.hash that passed back to datapath could only be set from the upcall->hash, i.e. the hash that received from the datapath during upcall.  Userspace never passes dp_hash that calculated by userspace to datapath, it only returns same hash that was calculated by datapath itself before upcall.17:44
zhouhanblp: ofproto/trace only tells slowpath needed, but didn't tell which action requires that.17:44
blpOK, it should be possible to figure it out. Do you have the trace handy?17:45
zhouhanimaximets: is it possible to pass it back (i.e. is it a small fix?)17:45
zhouhanFinal flow: icmp,reg11=0xe,reg12=0x18,reg14=0xa,reg15=0x11,tun_id=0xa0011ff0002,tun_src=10.172.66.12,tun_dst=10.78.211.43,tun_ipv6_src=::,tun_ipv6_dst=::,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=0,tun_ttl=0,tun_erspan_ver=0,tun_flags=csum|key,metadata=0xff0002,in_port=457,vlan_tci=0x0000,dl_src=aa:aa:bb:00:01:02,dl_dst=aa:aa:bb:00:03:01,nw_src=10.227.183.232,nw_dst=10.9.0.1,nw_tos=0,nw_ecn=0,nw_ttl=53,icmp_type=8,icmp_code=017:45
zhouhanMegaflow: recirc_id=0,eth,icmp,tun_id=0xa0011ff0002,tun_src=10.172.66.12,tun_dst=10.78.211.43,tun_tos=0,tun_flags=-df+csum+key,in_port=457,vlan_tci=0x0000/0x1000,dl_src=aa:aa:bb:00:01:02,dl_dst=aa:aa:bb:00:03:01,nw_src=10.227.183.232,nw_dst=10.9.0.1,nw_ttl=53,nw_frag=no,icmp_type=0x8/0xff,icmp_code=0x0/0xff17:45
zhouhanDatapath actions: ct_clear,ct_clear,ct_clear,set(eth(src=aa:aa:aa:00:01:01,dst=aa:aa:aa:00:00:01)),set(ipv4(src=10.9.0.1,dst=10.227.183.232,ttl=254)),set(icmp(type=0,code=0)),hash(l4(0)),recirc(0x3)17:45
zhouhanThis flow is handled by the userspace slow path because it:17:46
zhouhan  - Uses action(s) not supported by datapath.17:46
blpIt's not a good idea to pass a userspace-caluclated hash back to the kernel because the kernel would calculate a different value.17:46
zhouhanblp: This is the last part of trace17:46
imaximetszhouhan, I'm not sure.  It might be possible to check if we have no upcall->hash, but have dp_hash and pass dp_hash to execute.hash instead, but I'm not sure.17:46
zhouhanblp: imaximets: Then do you think it is a better idea to always do dp_hash in datapath?17:46
imaximetszhouhan, I think, yes, it's better than 'datapath hash' is calculated by datapath.17:47
zhouhan(sorry this might have taken too long, in case someone else want to update)17:47
blpIt's the set actions for ICMP that are doing it. The datapath doesn't know how to change ICMP.17:48
imaximetss/than/when/17:48
zhouhanblp: is it the ICMP fields setting require slowpath?17:49
blpzhouhan: yes17:49
zhouhanblp: ok, thanks!17:49
zhouhanblp: it would be better if trace can just point this out :)17:49
blpzhouhan: yes17:49
zhouhanother than this, I was involved in some discussions and also trying to fix some bugs in ovn.17:50
*** acidfoo_ has quit IRC17:50
zhouhanOne of the discussion was about ARP flows exploding in LRs. I think I can work out the configurably disable static ARP resolve in LR, which would solve the issue for ovn-k8s.17:51
zhouhanThat's my update :)17:51
*** acidfoo_ has joined #openvswitch17:51
_lore_zhouhan: regarding gateway flow issue, IIRC this chunks was to distribute non DVR traffic17:52
zhouhan_lore_: I got your point, but why was different prirority needed?17:52
_lore_I need to get back to it since I can't recall the details now17:53
zhouhan_lore_: ok, thanks!17:53
zhouhan_lore_: It seems an optimization, right?17:53
_lore_nope17:53
_lore_let's say you have FIP 192.168.1.117:54
_lore_this is to distribute traffic for 192.168.1.0/24 IIRC17:54
_lore_but I will check17:54
zhouhan_lore_: ok, I was thinking we could revert it, if it is an optimization and if we couldn't solve the route priority problem before the release17:54
_lore_what is the issue you are facing?17:55
zhouhan_lore_: If you see my example, the /16 route is overriding the /24 route, which is wrong17:55
_lore_ack17:56
_lore_I will look into it17:56
zhouhanthanks _lore_17:56
_lore_yw :)17:56
*** ryzhyk has quit IRC17:57
_lore_zhouhan: do you have a unit-test for it?17:57
zhouhan_lore_: here is what I did in a sandbox:17:59
zhouhan  989  ovn-nbctl lr-add lr117:59
zhouhan  990  ovn-nbctl lrp-add lr1 lrp1 aa:aa:aa:aa:aa:01 192.168.0.1/2417:59
zhouhan  991  ovn-nbctl lrp-add lr1 lrp2 aa:aa:aa:aa:aa:02 192.168.100.1/2417:59
zhouhan  992  ovn-nbctl lr-route-add lr1 10.0.0.0/24 192.168.0.217:59
zhouhan  993  ovn-nbctl lr-route-add lr1 10.0.0.0/16 192.168.100.217:59
zhouhan  994  ovn-sbctl lflow-list17:59
zhouhan  995  ovn-nbctl --help | grep gateway17:59
zhouhan  996  ovn-sbctl show17:59
zhouhan  997  ovn-nbctl lrp-set-gateway-chassis lrp1 chassis-117:59
zhouhan  998  ovn-sbctl lflow-list17:59
_lore_ok, thx17:59
_lore_I think we should add some unitest for it18:00
blp_lore_: please!18:02
_lore_will do :)18:02
*** acidfoo_ has quit IRC18:03
mmichelsonAnyone else want to take a turn at the mic?18:05
flaviofbye all!18:06
blpbye!18:09
dcearabye!18:10
mmichelson#endmeeting18:11
openstackMeeting ended Thu May 14 18:11:20 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:11
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-14-17.14.html18:11
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-14-17.14.txt18:11
openstackLog:            http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-14-17.14.log.html18:11
*** acidfoo_ has joined #openvswitch18:30
*** dceara has quit IRC18:53
*** anilvenkata has quit IRC19:17
*** slaweq has quit IRC19:51
*** slaweq has joined #openvswitch19:59
*** mmirecki has joined #openvswitch20:20
*** mmirecki has quit IRC20:44
*** gmg1 has quit IRC21:22
*** JamesBenson has quit IRC21:34
*** acidfoo_ has quit IRC21:40
*** slaweq has quit IRC22:02
*** tbachman_ has joined #openvswitch22:05
*** tbachman has quit IRC22:05
*** tbachman_ is now known as tbachman22:05
*** dceara has joined #openvswitch22:11
*** dceara has quit IRC22:28
*** acidfoo_ has joined #openvswitch22:45
*** tbachman_ has joined #openvswitch22:52
*** tbachman has quit IRC22:53
*** tbachman_ is now known as tbachman22:53
*** bostondriver has quit IRC23:04

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!