Thursday, 2017-06-01

tbarronbswartz: (1) implement https://github.com/openstack/manila-specs/blob/master/specs/pike/ensure-share.rst, so driver can choose to generate a new hash if it needs a refresh by e.g. returning {'access_list_epoch: <new value>} to get_backend_info() call from the manager.00:17
*** gaurangt has quit IRC00:20
tbarron(2) Add a new enum value for share_instance['access_rules_status'], say 'NEEDS_REFRESH', such that (a) driver can return {<share-insnace id: {'access_rules_status': NEEDS_REFRESH}, ...} to the manager's ensure_shares() call, and00:21
*** gaurangt has joined #openstack-manila00:22
tbarron(b) the manager can modify its logic around share_instance['access_rules_status'] to send a full refresh of rules in the case of NEEDS_REFRESH instead of only the current condition where00:22
tbarronaccess rules for a share are sent in case share_instance['access_rules_status'] != constants.STATUS_ACTIVE and there are pending APPLYIING or DENYING access rules.00:24
tbarrongouthamr: please sanity check the above at your leisure ^^^00:25
tbarronbswartz: I would in parallel be working on a backend shared state solution for cephfs/nfs-ganesha driver with the state persisted outside manila so that we shouldn't have to invoke such a refresh simply because we are doing a failover.00:27
tbarronbswartz: but we'd want the refresh as a fallback path and depending on timing of various deliverables might use it as a temporary stopgap.00:28
*** hongbin has quit IRC00:44
*** jmlowe has joined #openstack-manila00:51
*** markstur has quit IRC00:53
*** markstur has joined #openstack-manila00:54
*** markstur has quit IRC00:59
*** markstur has joined #openstack-manila01:46
*** markstur has quit IRC01:51
*** kaisers_ has joined #openstack-manila01:53
*** kaisers has quit IRC01:56
openstackgerritJohnnyChou proposed openstack/manila master: Add Infortrend Manila Driver  https://review.openstack.org/46584602:24
*** gaurangt_ has joined #openstack-manila03:41
*** gaurangt has quit IRC03:42
*** gaurangt_ has quit IRC03:50
*** gaurangt has joined #openstack-manila03:54
*** arnewiebalck_ has joined #openstack-manila04:12
*** kaisers_ has quit IRC04:37
*** pcaruana has joined #openstack-manila04:43
*** gouthamr has quit IRC04:55
*** gaurangt has quit IRC04:59
*** gaurangt has joined #openstack-manila05:00
*** arnewiebalck_ has quit IRC05:06
*** arnewiebalck_ has joined #openstack-manila05:07
*** arnewiebalck_ has quit IRC05:22
*** kaisers has joined #openstack-manila05:38
*** kaisers has quit IRC05:44
*** pcaruana has quit IRC05:49
*** pcaruana has joined #openstack-manila06:02
*** jprovazn has joined #openstack-manila06:18
*** gaurangt_ has joined #openstack-manila06:25
*** gaurangt has quit IRC06:25
*** kaisers has joined #openstack-manila06:41
*** gaurangt has joined #openstack-manila07:06
*** gaurangt_ has quit IRC07:06
*** td has quit IRC07:07
*** markstur has joined #openstack-manila07:12
*** gaurangt_ has joined #openstack-manila07:14
*** gaurangt has quit IRC07:15
*** kaisers has quit IRC07:15
*** markstur has quit IRC07:17
*** gaurangt has joined #openstack-manila07:18
*** gaurangt_ has quit IRC07:18
*** td has joined #openstack-manila07:19
*** markstur has joined #openstack-manila07:26
*** dsariel has joined #openstack-manila07:26
openstackgerritzhongjun proposed openstack/manila master: TEST  https://review.openstack.org/46940707:31
*** markstur has quit IRC07:31
*** gaurangt_ has joined #openstack-manila07:42
*** gaurangt has quit IRC07:43
openstackgerritzhongjun proposed openstack/manila master: [WIP] Change ensure share to make startup faster  https://review.openstack.org/45754507:48
*** gaurangt has joined #openstack-manila07:52
*** gaurangt_ has quit IRC07:53
*** gaurangt_ has joined #openstack-manila07:59
*** gaurangt has quit IRC08:01
openstackgerritzhongjun proposed openstack/manila master: [WIP] Change ensure share to make startup faster  https://review.openstack.org/45754508:03
*** kaisers has joined #openstack-manila08:12
*** gaurangt has joined #openstack-manila08:14
*** gaurangt_ has quit IRC08:17
*** kaisers has quit IRC08:17
openstackgerritJan Provaznik proposed openstack/manila master: User Messages  https://review.openstack.org/31354908:17
openstackgerritJan Provaznik proposed openstack/manila master: Provide filter name in user messages  https://review.openstack.org/44310208:17
openstackgerritJan Provaznik proposed openstack/manila master: Extend usage of user messages  https://review.openstack.org/44310108:17
*** gaurangt_ has joined #openstack-manila08:29
*** gaurangt has quit IRC08:30
*** a-pugachev has joined #openstack-manila08:52
*** markstur has joined #openstack-manila09:03
*** gaurangt has joined #openstack-manila09:04
*** gaurangt_ has quit IRC09:05
*** markstur has quit IRC09:08
*** gaurangt has quit IRC09:09
*** gaurangt has joined #openstack-manila09:11
*** kaisers has joined #openstack-manila09:14
*** gaurangt has quit IRC09:20
*** gaurangt has joined #openstack-manila09:21
zhongjuntbarron: Hi,  You want to use ensure_share(s) to force your backend to re-create access rules for shares. It sounds like it is another part when we discuss about refactor ensure share: "weather we can decide which resources(snapshot, access, group) we need to update".  If we allow driver to ask ensure_share to force  your backend to re-create access rules09:33
zhongjunfor shares. How about other resources?09:33
*** openstackstatus has quit IRC09:36
*** openstackstatus has joined #openstack-manila09:38
*** ChanServ sets mode: +v openstackstatus09:38
-openstackstatus- NOTICE: There is a known issue with setuptools 36.0.0 and errors about the "six" package. For current details see https://github.com/pypa/setuptools/issues/1042 and monitor #openstack-infra09:44
*** kaisers has quit IRC09:49
zhongjuntbarron: If we need to recreate share access in ensure share.  We could just get the flag( {'access_list_epoch: <new value>} ) return by driver function( get_backend_info) and refresh access if the flag is true(means need to update    share_instance['access_rules_status'] != constants.STATUS_ACTIVE or flag=True ). If we have the access_update_flag or09:49
zhongjuninstance_update_flag, we could just update access and do not update share instance. It could not need to  add a new enum value for share_instance['access_rules_status'], say 'NEEDS_REFRESH'.09:49
*** gaurangt_ has joined #openstack-manila09:52
*** gaurangt has quit IRC09:52
tbarronzhongjun: yes, we could have a specific flag for refreshing access lists (and I guess specific flags for updating snapshots, groups, etc. but I don't need those atm).10:30
tbarronzhongjun: instead of returning an access_rules_status that would trigger the refresh10:31
tbarronzhongjun: thank you for thinking about the problem10:31
openstackgerritzhongjun proposed openstack/manila master: [WIP] Change ensure share to make startup faster  https://review.openstack.org/45754510:39
zhongjuntbarron: It is my pleasure.10:40
*** kaisers has joined #openstack-manila10:46
*** ganso has joined #openstack-manila10:46
*** markstur has joined #openstack-manila11:04
gansotbarron: hey Tom! I just read about yesterday's discussion on access rules11:05
tbarronganso: great, thanks!  What are your thoughts?11:05
gansotbarron: I like your idea, when Goutham and I removed the recovery mode we did so because it was no longer needed in the new design, but we also did not consider the use case you described. I think using ensure_share as you described is the way to go :)11:06
gansotbarron: but I have a few questions11:06
tbarronganso: of course11:07
gansotbarron: when you failover, don't you failover from backendA to backendB, which are both visible and managed by manila?11:07
*** kaisers has quit IRC11:08
tbarronganso: no, this is a pacemaker controlled failover from one node to another where processes running on multiple nodes are just11:08
tbarronstandby until a failover, in which case they become active and fill in the "slot" for the same backend11:09
*** markstur has quit IRC11:09
tbarronganso: we're already doing this for manila share in general, with whatever backends are configured for it.11:09
gansotbarron: oh I see, so this backend remains not seen by manila and takes over backendA's role when it fails. ok good11:09
gansotbarron: I like the fact that we would be setting a flag like "NEEDS_REFRESH", it is something that is controlled on our side11:10
tbarronganso: the new thing here is that there's a ganesha process that also fails over that is the backend itself.11:10
tbarron(or part of it :))11:10
gansotbarron: so, how does manila know something is failing over? how do you intend to check the need to set the "NEEDS_REFRESH" flag?11:12
tbarronganso: well use cases will probably be different for different backends.  The driver would set it.  In our case I think we can know that pacemaker started us up because of a failover.11:15
tbarronSo we know that our saved access rule state (if we see that we have it) is probably stale.11:15
* tbarron notes that we're working in parallel to persist that state on shared storage outside manila so that wouldn't be the case.11:16
gansotbarron: hummm, it is a bit unusual workflow to allow a driver to set a DB entry flag... we would need a driver interface "check_failover" that returns a boolean and manager will change DB and then invoke ensure_share for that share. Or something along these lines.11:16
tbarronBut even when we solve that perhaps the shared state is no longer accessible for some reason and needs to be rebuilt.11:17
gansotbarron: ^ in the example I just described the flag is really not needed.11:17
tbarronganso: the ensure_shares spec already allows driver to cause DB update, so it could be done the same way as export locations and share state in the example in that spec11:17
tbarronganso: no extra driver interface would be needed since that spec provides a generic mechanism for doing this via get_backend_info()11:19
gansotbarron: ensure share will be invoked when changes are detected. Unless the failover is detected in this step (this would be the cleanest approach and less code change), you would need another step11:19
gansotbarron: unless the failover is *not detected, you would need another step11:19
gansotbarron: if so, again, there is no need for a NEEDS_REFRESH flag11:20
gansotbarron: this looks to be much simpler than I initially thought11:20
tbarronganso: agree, that's why my suggestion was (for one way to do this) that when the driver comes up if it decides a refresh is needed, it would add a key-value (doesn't really matter much what it is but I was calling it 'access-sync-epoch' or something like that) that causes the hash to change.11:21
tbarronganso: But I think there are multiple ways we could do this.  I was mostly checking that we agreed that ensure_shares() should be able to refresh access-rule state as well as bare share state.11:22
tbarronganso: it seems like we all agree that this should be something that we make possible as we implement ensure_share() though bswartz has made the valid point11:23
tbarronthat even if we make it possible we should be clear that it's an expensive operation that scales badly so that drivers should avoid using it unless absolutely necessary.11:24
gansotbarron: yes, I agree. we should not always refresh the access rules in ensure_share (given ensure_share is being invoked)11:26
tbarronganso: thanks for thinking about this!11:39
gansotbarron: np! I am glad to be helping :)11:44
*** MVenesio has joined #openstack-manila11:56
*** kaisers has joined #openstack-manila12:01
*** kaisers has quit IRC12:23
*** dsariel has quit IRC12:29
*** rraja has joined #openstack-manila12:46
*** markstur has joined #openstack-manila12:55
*** markstur has quit IRC13:00
*** gouthamr has joined #openstack-manila13:01
*** dsariel has joined #openstack-manila13:19
*** kaisers has joined #openstack-manila13:20
*** kaisers has quit IRC13:25
bswartzmkoderer: question about neutron port binding plugin13:36
bswartzmkoderer: it seems to work great when we use tenant_network_type = vlan, but not when we use vxlan13:36
*** dsariel has quit IRC13:37
*** eharney has joined #openstack-manila13:43
*** eharney has quit IRC14:00
openstackgerritValeriy Ponomaryov proposed openstack/manila master: Add quotas per share type  https://review.openstack.org/45215814:05
openstackgerritTom Barron proposed openstack/manila master: Fix pep8 M325 error with python 3.5  https://review.openstack.org/46952314:11
*** eharney has joined #openstack-manila14:15
*** dsariel has joined #openstack-manila14:18
*** kaisers has joined #openstack-manila14:21
*** cknight has joined #openstack-manila14:22
*** xyang_ has joined #openstack-manila14:24
*** jprovazn has quit IRC14:29
-openstackstatus- NOTICE: python-setuptools 36.0.1 has been released and now making its way into jobs. Feel free to 'recheck' your failures. If you have any problems, please join #openstack-infra14:33
*** openstackgerrit has quit IRC14:34
gouthamrbswartz: https://i.redd.it/tfvrt0s12z0z.png14:43
*** openstackgerrit has joined #openstack-manila14:43
openstackgerritzhongjun proposed openstack/python-manilaclient master: Change to share access list API  https://review.openstack.org/46828714:43
*** kaisers has quit IRC14:44
*** hongbin has joined #openstack-manila14:47
openstackgerritzhongjun proposed openstack/python-manilaclient master: Change to share access list API  https://review.openstack.org/46828714:50
gansogouthamr: lol14:52
openstackgerritAlex Schultz proposed openstack/puppet-manila master: Drop -dev in preparation for Pike M2  https://review.openstack.org/46989614:54
*** markstur has joined #openstack-manila14:59
*** rraja has quit IRC15:01
*** eharney has quit IRC15:05
*** eharney has joined #openstack-manila15:18
*** toabctl has quit IRC15:22
*** toabctl has joined #openstack-manila15:23
*** openstackgerrit has quit IRC15:33
tbarronbswartz: there is no longer a separate unit file for manila api so the instead of 'systemctl restart openstack-manila-api' (my target isn't just right) one does 'kill -SIGINT <pid>'15:38
tbarronand instead of 'journalctl -f --unit=<unitfile-target>' I think one can do 'journalctl SYSLOG_IDENTIFIER=manila-api' (except that for some reason that isn't working in RDO currently).15:41
*** kaisers has joined #openstack-manila15:42
tbarronIMO it was nice to be able to use the same systemctl and journalctl commands for api and non-api services, but15:42
tbarronI'm not arguing that "nice" outweighs the other considerations like lack of eventlet dependency, port consolidation, etc.15:43
tbarronI just wouldn't be surprised if there's some backlash from cloud operators ...15:44
*** MVenesio has quit IRC15:46
*** cknight has quit IRC15:47
bswartztbarron: the exact systemd configuration will be entirely up to the distros I think15:57
gouthamrtbarron: it's still whatever RH/OOO decides to deploy the api service like, isn't it?15:57
gouthamrtbarron: would they be giving a choice to deployers too?15:57
gouthamrs/they/you15:57
tbarrongouthamr: TBD15:57
tbarronbswartz: agree15:57
bswartzif Redhat makes it that way then it's RedHat's fault if the customers don't like it -- upstream doesn't control how distros configure services15:57
bswartzthe way I see it, we're providing one more tool15:58
gouthamrtbarron: i like persistent uncompressed logs.. but then i don't run production clouds at scale..15:58
tbarronMy main question was just whether we have the same set of issues with the alternate web server from zhongjun's patch.15:58
bswartzgouthamr: it's not hard to make systemd produce those15:58
gouthamrbswartz: yes, i'd hope OOO/Packstack would configure all my services that way15:59
bswartztbarron: I don't really know anything about uwsgi15:59
bswartzgouthamr: packstack already does15:59
gouthamrcontinue to.15:59
bswartzI don't see why it would change15:59
tbarronbswartz: I didn't mean to come off as saying that this is an upstream problem, was just wondering about uwsgi and raising awareness a bit within devstack15:59
bswartzI leave these decisions to smarter people than me16:00
tbarronas a developer with devstack I was thrown a bit b/c the systemd by default stuff arrived more or less at the same time as apache-by-default16:00
bswartzwhen I grew up Apache was your only choice16:00
tbarronso as goutham knows I was going "where are my api logs and how do I restart manila-api to set DEBUG, etc."16:01
*** winston-d_ has joined #openstack-manila16:01
tbarronscreen went away and systemctl/journalctl didn't work16:01
tbarronfor api16:01
*** catintheroof has joined #openstack-manila16:02
gouthamrwe could have gone the route of making sure the logs were still emitted to LOG_DIR16:03
gouthamr:)16:03
gouthamrinstead of apache's LOG_DIR16:03
tbarronRDO is putting the logs still, uncompressed, under /var/log/{manila,cinder,...}/*api.log16:03
bswartz:-D16:03
tbarronYou don't have to use journalctl to look at them.16:03
tbarronBut it's nice to use journalctl with wild card on all the manila unit files and that now doesn't include the api log.16:04
gouthamrsure, but i might have something else parse uncompressed log files...16:05
gouthamrs/but/and (agreeing)16:05
*** kaisers has quit IRC16:05
tbarrongouthamr: right, I agree that one should be able to see the logs w/o journalctl too.16:05
tbarronIt was also nice to be able to say 'systemctl restart openstack-manila*'16:06
tbarronYou can still say it, it just won't restart the api service.16:06
*** openstackgerrit has joined #openstack-manila16:07
openstackgerritMerged openstack/puppet-manila master: Drop -dev in preparation for Pike M2  https://review.openstack.org/46989616:07
gouthamrso if multiple services are configured as apache vhosts, should I be reloading apache to grab changes to configuration16:08
bswartzgouthamr: yes16:08
bswartzin a dev environment bouncing apache is harmless16:09
bswartzI think tbarron's point is that simply bouncing apache to pick up a manila-api config change might be too disruptive16:09
gouthamryes16:10
tbarronso we 'kill -SIGINT <pid>' for the apache pid running manila-api16:10
bswartzI mean in a production environment16:11
tbarronbswartz: yeah, I'm just saying what we will recommend in production env to avoid bouncing all the api services and horizon just to restart manila-api16:11
tbarronin non-production env who cares16:12
gouthamr(systemctl reload httpd)'s behavior is a graceful restart; so not disruptive, but potential delay/painpoint to other traffic16:12
gouthamrokay, don't think that's right, according to https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/ch-Web_Servers.html16:14
tbarrongouthamr: it says it's a graceful restart that causes reload of apache conf file.  I'll have to test whether it causes reload of manila.conf, cinder.conf, etc.16:16
tbarrongouthamr: it does, so the situation isn't so bad as I thought: one can use reload and it's not so alarming, though every api service "pauses" from end user standpoint.16:20
*** dsariel has quit IRC16:21
openstackgerritMerged openstack/manila master: Fix pep8 M325 error with python 3.5  https://review.openstack.org/46952316:25
*** catintheroof has quit IRC16:27
*** cknight has joined #openstack-manila16:53
*** kaisers has joined #openstack-manila17:02
*** a-pugachev has quit IRC17:16
gouthamrtbarron: yes.. that's what i'd assume, and i think that's bad to an unsuspecting client17:19
tbarrongouthamr: ack17:20
*** kaisers has quit IRC17:26
*** kaisers has joined #openstack-manila17:28
*** kaisers has quit IRC17:32
*** MVenesio has joined #openstack-manila17:35
*** kaisers has joined #openstack-manila17:45
openstackgerritOpenStack Proposal Bot proposed openstack/manila master: Updated from global requirements  https://review.openstack.org/46957017:53
*** xyang_ has quit IRC18:01
*** xyang_ has joined #openstack-manila18:03
*** xyang_ has quit IRC18:04
openstackgerritValeriy Ponomaryov proposed openstack/manila master: Add quotas per share type  https://review.openstack.org/45215818:21
openstackgerritMerged openstack/manila master: Use get_notification_transport for notifications  https://review.openstack.org/46963018:29
*** xyang1 has joined #openstack-manila19:08
*** xyang_ has joined #openstack-manila19:21
*** MVenesio has quit IRC19:30
*** eharney has quit IRC20:03
*** winston-d_ has quit IRC20:39
*** xyang_ has quit IRC20:47
*** xyang_ has joined #openstack-manila20:47
*** cknight has quit IRC20:49
*** kaisers1 has quit IRC21:12
openstackgerritTom Barron proposed openstack/manila master: Clean releasenotes and install-guide build dir  https://review.openstack.org/47001921:20
*** kaisers1 has joined #openstack-manila21:26
openstackgerritOpenStack Proposal Bot proposed openstack/manila master: Updated from global requirements  https://review.openstack.org/46957021:32
*** xyang_ has quit IRC21:57
*** xyang_ has joined #openstack-manila21:59
*** markstur has quit IRC22:00
*** markstur has joined #openstack-manila22:05
*** gouthamr has quit IRC22:06
openstackgerritOpenStack Proposal Bot proposed openstack/manila master: Updated from global requirements  https://review.openstack.org/46957022:12
*** xyang_ has quit IRC22:20
*** ganso has quit IRC22:26
*** gouthamr has joined #openstack-manila22:58
*** markstur has quit IRC23:35
*** xyang1 has quit IRC23:44
*** markstur has joined #openstack-manila23:46
*** hongbin has quit IRC23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!