Friday, 2017-02-17

*** hongbin has quit IRC00:35
*** tovin07 has joined #openstack-containers00:39
*** EricGonc_ has quit IRC00:43
*** jwcroppe has quit IRC00:57
*** juggler has joined #openstack-containers01:00
*** jberkus has quit IRC01:05
*** kevinz has joined #openstack-containers01:08
*** catintheroof has quit IRC01:13
*** edisonxiang has quit IRC01:19
*** edisonxiang has joined #openstack-containers01:19
*** ricolin has joined #openstack-containers01:20
*** salmankhan has quit IRC01:21
*** mtanino has quit IRC01:22
*** Syed__ has quit IRC01:25
*** dave-mccowan has joined #openstack-containers01:27
*** sdake_ has joined #openstack-containers01:27
*** sdake has quit IRC01:30
*** jberkus has joined #openstack-containers02:03
*** tonanhngo has quit IRC02:06
*** dfflanders has quit IRC02:09
*** dfflanders has joined #openstack-containers02:10
*** tonanhngo has joined #openstack-containers02:12
*** tonanhngo has quit IRC02:17
*** zz_dimtruck is now known as dimtruck02:19
*** EricGonczer_ has joined #openstack-containers02:32
*** ianychoi has quit IRC02:34
*** juggler has quit IRC02:35
*** ianychoi has joined #openstack-containers02:38
*** EricGonc_ has joined #openstack-containers02:39
*** EricGonczer_ has quit IRC02:40
*** vijaykc4 has joined #openstack-containers02:44
*** wangbo has joined #openstack-containers02:58
*** EricGonc_ has quit IRC03:03
*** vijaykc4 has quit IRC03:12
*** vijaykc4 has joined #openstack-containers03:15
*** vijaykc4 has quit IRC03:23
*** wangbo has quit IRC03:25
*** sdake_ has quit IRC03:36
*** sdake has joined #openstack-containers03:39
*** hongbin has joined #openstack-containers03:43
*** deepbook5broo has joined #openstack-containers03:43
*** deepbook5broo has left #openstack-containers03:44
*** dave-mccowan has quit IRC03:47
*** zhugaoxiao has quit IRC03:49
*** david-lyle has quit IRC03:49
*** zhugaoxiao has joined #openstack-containers03:50
*** david-lyle has joined #openstack-containers03:50
*** jberkus has quit IRC03:53
*** sdake_ has joined #openstack-containers03:53
*** sdake has quit IRC03:55
openstackgerritAnh Tran proposed openstack/magnum master: Remove unused logging import  https://review.openstack.org/43521903:57
*** adrian_otto has joined #openstack-containers03:58
*** janki has joined #openstack-containers04:09
*** adrian_otto has quit IRC04:14
*** hongbin has quit IRC04:22
*** dimtruck is now known as zz_dimtruck04:24
*** adrian_otto has joined #openstack-containers04:34
*** adisky_ has joined #openstack-containers04:38
*** v1k0d3n has quit IRC04:40
*** mdnadeem has joined #openstack-containers04:40
*** daidv has joined #openstack-containers04:46
*** v1k0d3n has joined #openstack-containers04:52
*** adrian_otto has quit IRC04:54
*** sdake_ has quit IRC04:54
*** ricolin has quit IRC04:59
*** v1k0d3n has quit IRC05:05
*** rcernin has joined #openstack-containers05:08
*** feke has quit IRC05:14
*** maestropandy has joined #openstack-containers05:17
*** v1k0d3n has joined #openstack-containers05:18
*** faizy has joined #openstack-containers05:32
*** vijaykc4 has joined #openstack-containers05:43
*** daidv has quit IRC05:54
*** vijaykc4 has quit IRC05:56
*** daidv has joined #openstack-containers06:00
*** vijaykc4 has joined #openstack-containers06:02
openstackgerritOpenStack Proposal Bot proposed openstack/magnum-ui master: Imported Translations from Zanata  https://review.openstack.org/43525906:05
*** vijaykc4 has quit IRC06:07
openstackgerritfengbeihong proposed openstack/magnum master: Support dcos installation on centos vm cluster  https://review.openstack.org/38643506:08
*** yamamoto has quit IRC06:08
openstackgerritfengbeihong proposed openstack/magnum master: Support dcos installation on centos vm cluster  https://review.openstack.org/38643506:11
*** jberkus has joined #openstack-containers06:11
*** sidx64 has joined #openstack-containers06:13
*** jberkus has quit IRC06:15
*** rcernin has quit IRC06:21
*** vijaykc4 has joined #openstack-containers06:27
openstackgerritMichael Liu proposed openstack/magnum master: magnum support cni network driver  https://review.openstack.org/43527006:36
*** yuanying has quit IRC06:37
*** jerrygb has joined #openstack-containers06:41
*** jerrygb has quit IRC06:46
*** yamamoto has joined #openstack-containers06:47
*** sidx64_ has joined #openstack-containers06:47
*** sidx64 has quit IRC06:50
*** sidx64_ is now known as sidx6406:50
*** jerrygb has joined #openstack-containers07:00
*** rcernin has joined #openstack-containers07:01
*** askb has quit IRC07:04
*** vijaykc4 has quit IRC07:05
*** jerrygb has quit IRC07:05
*** Oku_OS-away is now known as Oku_OS07:07
*** askb has joined #openstack-containers07:15
*** fragatina has quit IRC07:19
*** dsariel has quit IRC07:26
*** fragatina has joined #openstack-containers07:35
*** fragatina has quit IRC07:37
*** fragatina has joined #openstack-containers07:38
*** code34_ has joined #openstack-containers07:38
*** code34_ has quit IRC07:38
*** fragatina has quit IRC07:38
*** dsariel has joined #openstack-containers07:39
*** fragatina has joined #openstack-containers07:39
openstackgerrityuhui proposed openstack/magnum master: Fix some typos  https://review.openstack.org/43528307:40
*** maestropandy has quit IRC07:45
*** amoralej|off is now known as amoralej07:46
*** yuanying has joined #openstack-containers07:57
*** hishh has joined #openstack-containers08:04
openstackgerritKevin Lefevre proposed openstack/magnum master: Prepare Kubelet for multiple container runtime  https://review.openstack.org/43492108:10
*** hishh has quit IRC08:17
*** yamamoto has quit IRC08:18
*** pcaruana has joined #openstack-containers08:22
*** pcastets has joined #openstack-containers08:29
*** yamamoto has joined #openstack-containers08:31
*** vijaykc4 has joined #openstack-containers08:41
*** yamamoto has quit IRC08:44
*** vimal has joined #openstack-containers08:45
*** code34 has quit IRC08:45
*** daidv has quit IRC08:46
*** Serlex has joined #openstack-containers08:53
*** ssbarnea has joined #openstack-containers08:56
ArchiFleKshi, could someone take a look a this bug https://bugs.launchpad.net/magnum/+bug/1643197 I think it is a real issue that should be addressed as it can "break" cluster creation completly when instances cannot access the internalURL Api network (which is the case for almost every production deployment I think)09:00
openstackLaunchpad bug 1643197 in Magnum "os_magnum role uses internal VIP inside instances" [Undecided,New]09:00
openstackgerrityatin proposed openstack/python-magnumclient master: Compare test coverage with the master branch  https://review.openstack.org/43531609:01
*** tovin07 has quit IRC09:26
*** maestropandy has joined #openstack-containers09:30
*** maestropandy has left #openstack-containers09:30
*** Jeffrey4l has quit IRC09:42
*** yamamoto has joined #openstack-containers09:44
*** vimal has quit IRC09:49
*** yamamoto has quit IRC09:49
*** vimal has joined #openstack-containers09:49
*** Jeffrey4l has joined #openstack-containers09:55
*** vijaykc4 has quit IRC09:55
*** vijaykc4 has joined #openstack-containers09:55
*** salmankhan has joined #openstack-containers09:56
*** Jeffrey4l has quit IRC09:56
*** Jeffrey4l has joined #openstack-containers09:57
*** ssbarnea has quit IRC10:02
*** ssbarnea has joined #openstack-containers10:03
strigaziArchiFleKs I'll have a look again this afternoon10:08
*** faizy has quit IRC10:12
*** faizy has joined #openstack-containers10:13
*** kevinz has quit IRC10:14
*** yamamoto has joined #openstack-containers10:17
*** Jeffrey4l has quit IRC10:18
*** vijaykc4 has quit IRC10:18
*** Jeffrey4l has joined #openstack-containers10:19
*** vijaykc4 has joined #openstack-containers10:19
*** pcastets has quit IRC10:21
*** yamamoto has quit IRC10:25
*** sidx64 has quit IRC10:27
*** vijaykc4 has quit IRC10:28
*** sidx64 has joined #openstack-containers10:29
*** pcastets has joined #openstack-containers10:35
*** jerrygb has joined #openstack-containers10:46
*** jerrygb has quit IRC10:50
*** jerrygb has joined #openstack-containers10:51
*** jerrygb_ has joined #openstack-containers10:54
*** jerrygb has quit IRC10:55
*** vijaykc4 has joined #openstack-containers11:04
*** sidx64_ has joined #openstack-containers11:05
*** sidx64 has quit IRC11:08
*** sidx64_ is now known as sidx6411:08
*** sdake has joined #openstack-containers11:18
*** itoperatorguy has joined #openstack-containers11:24
itoperatorguyhi all11:24
itoperatorguycan I ask a fast magnum issue question here?11:25
itoperatorguyI have installed magnum followed by official documentation. I created a swarm cluster using the fedora-atomic-latest image. I configured the env variables using the $(magnum cluster-config swarm-cluster --dir working_dir) command. But when I want to test with 'docker run busybox echo "HEllo From docker"' then I get docker: Error response from daemon: No healthy node available in the cluster. If I am ssh into the swarm master VM11:31
itoperatorguyor the 'slave' node vm then the docker run command works without issue. What do I miss? (using tcpdump on the swarm master VM i see that packets arriving to the port 2376 when I am running docker run from controller1 node)11:31
*** salmankhan has quit IRC11:33
*** salmankhan has joined #openstack-containers11:34
*** vijaykc4 has quit IRC11:34
*** vijaykc4 has joined #openstack-containers11:35
*** yolanda has quit IRC11:36
*** yamamoto has joined #openstack-containers11:37
*** yamamoto has quit IRC11:37
*** yolanda has joined #openstack-containers11:39
*** sidx64 has quit IRC11:43
*** sidx64 has joined #openstack-containers11:45
*** sdake has quit IRC11:49
*** sdake has joined #openstack-containers11:49
*** askb has quit IRC12:00
*** danil has joined #openstack-containers12:07
*** adisky_ has quit IRC12:09
*** jerrygb_ has quit IRC12:10
*** danil has quit IRC12:11
*** dave-mccowan has joined #openstack-containers12:13
*** pcastets has quit IRC12:19
strigaziitoperatorguy which version of magnum?12:21
*** catintheroof has joined #openstack-containers12:23
*** yamamoto has joined #openstack-containers12:28
*** jerrygb has joined #openstack-containers12:29
*** sdake has quit IRC12:39
*** jerrygb_ has joined #openstack-containers12:45
*** faizy has quit IRC12:45
*** jerrygb has quit IRC12:46
*** sdake has joined #openstack-containers12:46
itoperatorguySorry I was out for lunch. root@controller1 (adminv3) $> magnum --version12:57
itoperatorguy2.3.112:57
*** yamamoto has quit IRC12:58
*** mtanino has joined #openstack-containers12:59
*** sdake_ has joined #openstack-containers12:59
*** yamamoto has joined #openstack-containers13:01
*** jerrygb_ has quit IRC13:02
*** jerrygb has joined #openstack-containers13:02
*** sdake has quit IRC13:02
*** yamamoto has quit IRC13:06
strigazithat is the client, for server do pip freeze | grep magnum or dpkg -l | grep magnum or rpm -qa | grep magnum depending on your system13:07
*** pcastets has joined #openstack-containers13:09
itoperatorguyopenstack-magnum-api-3.1.1-1.el7.noarch13:11
itoperatorguyopenstack-magnum-common-3.1.1-1.el7.noarch13:11
itoperatorguypython2-magnumclient-2.3.1-1.el7.noarch13:11
itoperatorguyopenstack-magnum-conductor-3.1.1-1.el7.noarch13:11
itoperatorguypython-magnum-3.1.1-1.el7.noarch13:11
strigaziyou must use fedora-atomic-newton13:11
itoperatorguythanks. I will run the test and get back to you13:14
*** sdake has joined #openstack-containers13:14
*** mtanino has quit IRC13:15
*** sdake_ has quit IRC13:15
*** EricGonczer_ has joined #openstack-containers13:16
*** yuanying has quit IRC13:18
*** pcastets has quit IRC13:19
*** pcastets has joined #openstack-containers13:22
*** EricGonc_ has joined #openstack-containers13:24
*** salmankhan has quit IRC13:24
*** EricGonczer_ has quit IRC13:24
*** sidx64 has quit IRC13:32
*** faizy has joined #openstack-containers13:41
*** EricGonc_ has quit IRC13:47
*** salmankhan has joined #openstack-containers14:05
*** yamamoto has joined #openstack-containers14:05
*** mdnadeem has quit IRC14:09
*** yamamoto has quit IRC14:11
*** mtanino has joined #openstack-containers14:28
*** randallburt has joined #openstack-containers14:30
*** yatinkarel has quit IRC14:33
*** amoralej is now known as amoralej|lunch14:41
*** jwcroppe has joined #openstack-containers14:43
*** zz_dimtruck is now known as dimtruck14:44
*** yatinkarel has joined #openstack-containers14:45
openstackgerritSpyros Trigazis proposed openstack/magnum stable/newton: IGNORE: tests stable/newton jobs  https://review.openstack.org/43548014:56
*** jerrygb has quit IRC14:59
*** jerrygb has joined #openstack-containers15:00
itoperatorguy<strigazi> this time cluster is constantly in "CREATE_IN_PROGRESS" state. And when I run openstack server show I only see the swarm_master server and slave node does not come up15:00
itoperatorguymagnum-conductor.log does not really help: 2017-02-17 14:59:04.533 13333 WARNING oslo.service.loopingcall [-] Function 'magnum.conductor.handlers.cluster_conductor.HeatPoller.poll_and_check' run outlasted interval by 2.30 sec15:01
*** janki has quit IRC15:01
*** hongbin has joined #openstack-containers15:02
*** jerrygb has quit IRC15:04
*** yatinkarel has quit IRC15:15
*** amoralej|lunch is now known as amoralej15:18
*** faizy has quit IRC15:22
*** yatinkarel has joined #openstack-containers15:27
strigazidims We need to release the client today?15:28
ttxyou have a release request already$15:30
*** EricGonczer_ has joined #openstack-containers15:31
*** faizy has joined #openstack-containers15:32
dimsstrigazi : see review https://review.openstack.org/#/c/435241/15:35
*** vijaykc4 has quit IRC15:40
dimsstrigazi : just edit that review and add stable/ocata in there15:41
*** jvgrant__ has joined #openstack-containers15:42
strigazidims done15:43
dimsthanks strigazi15:44
*** jvgrant_ has quit IRC15:45
*** yatinkarel has quit IRC15:49
*** kbyrne has joined #openstack-containers15:52
strigaziitoperatorguy before was CREATE_COMLETE?15:52
*** Syed__ has joined #openstack-containers15:52
itoperatorguyyes with fedora-atomic-latest creation was complete15:53
itoperatorguystrigazi yes with fedora-atomic-latest creation was complete15:53
itoperatorguythis time it just stuck/hangs in the 'CREATE_IN_PROGRESS' state15:54
strigazican you go in the master and do docker ps15:55
strigazican you go in the master and do docker ps -s15:55
strigazican you go in the master and do docker ps -a15:55
strigazito see if the swarm manager is running15:55
strigaziand can you try a k8s cluster too?15:55
itoperatorguyI am not able to login to the swarm_master node this time (with image latest it worked)15:57
itoperatorguyI am not able to even ping its floating ip15:58
itoperatorguyit seems it is not properly booting up15:58
strigazitry to boot only a vm with the image16:00
strigazisha256sum should be 53998ae262d47435b02394268585770d253b21b3507bf8c3ec7f3f10dc89c65716:00
*** Drago1 has joined #openstack-containers16:01
*** yatinkarel has joined #openstack-containers16:02
*** kbyrne has quit IRC16:04
*** sdake has quit IRC16:04
itoperatorguysum is correct :(16:05
*** sdake has joined #openstack-containers16:06
*** kbyrne has joined #openstack-containers16:07
*** chris_hultin|AWA is now known as chris_hultin16:08
*** randallburt has quit IRC16:11
*** randallburt has joined #openstack-containers16:11
*** swatson_ has joined #openstack-containers16:14
itoperatorguystrigaz Trying to launch the instance I am getting  Failed to perform requested operation on instance "test-instance", the instance has an error status: Please try again later [Error: Build of instance 972ee124-f1d3-4ff4-86a9-b6ba57af5da8 aborted: Block Device Mapping is Invalid.].16:15
strigaziexcellent :(16:16
yatinkarel:)16:16
*** vijaykc4 has joined #openstack-containers16:19
yatinkarelitoperatorguy, nova boot command?16:20
*** pcaruana has quit IRC16:21
itoperatorguyI did it from dashboard. Now I am readding the image to glance. After that I will try with nova boot16:21
yatinkarelitoperatorguy, Ok16:21
yatinkarelwhen you created cluster, what was the state of swarm-master in openstack server list16:23
itoperatorguystatus was "ACTIVE" but I was unable to ssh or ping (when I used fedora-atomic-latest ping and ssh worked and both master and node vm started)16:25
*** vijaykc4 has quit IRC16:25
yatinkarelitoperatorguy, did you checked nova console-log <instance id>16:28
itoperatorguyyes and it seemed to me that the boot procedure stucked somewhere16:29
itoperatorguythere was no login prompt16:29
itoperatorguythis time manual start: openstack server create --image fedora-atomic-newton --flavor magnum.test --security-group 5ab61dec-3f90-4436-b0bf-a808ba5ca885 --key-name mykey --nic net-id=1b6ae2d1-ecfe-4ec5-9933-ddc776135567 test-vm16:31
itoperatorguyerror message: No valid host was found. There are not enough hosts available.16:32
itoperatorguyso error message is different and I have 2 compute node with enough free resources16:32
yatinkarelnova service-list16:32
itoperatorguyroot@controller1 (adminv3) $> nova service-list16:33
itoperatorguy+----+------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+16:33
itoperatorguy| Id | Binary           | Host                        | Zone     | Status  | State | Updated_at                 | Disabled Reason |16:33
itoperatorguy+----+------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+16:33
itoperatorguy| 1  | nova-cert        | controller1.openstack.local | internal | enabled | up    | 2017-02-17T16:33:43.000000 | -               |16:33
itoperatorguy| 2  | nova-consoleauth | controller1.openstack.local | internal | enabled | up    | 2017-02-17T16:33:43.000000 | -               |16:33
itoperatorguy| 5  | nova-scheduler   | controller1.openstack.local | internal | enabled | up    | 2017-02-17T16:33:42.000000 | -               |16:33
itoperatorguy| 6  | nova-conductor   | controller1.openstack.local | internal | enabled | up    | 2017-02-17T16:33:49.000000 | -               |16:33
itoperatorguy| 7  | nova-compute     | compute1.openstack.local    | nova     | enabled | up    | 2017-02-17T16:33:42.000000 | -               |16:33
itoperatorguy| 8  | nova-compute     | compute2.openstack.local    | nova     | enabled | up    | 2017-02-17T16:33:48.000000 | -               |16:33
itoperatorguy+----+------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+16:33
yatinkarelnova hypervisor-stats16:33
itoperatorguyroot@controller1 (adminv3) $> nova hypervisor-stats16:34
itoperatorguy+----------------------+-------+16:34
itoperatorguy| Property             | Value |16:34
itoperatorguy+----------------------+-------+16:34
itoperatorguy| count                | 2     |16:34
itoperatorguy| current_workload     | 0     |16:34
itoperatorguy| disk_available_least | 60    |16:34
itoperatorguy| free_disk_gb         | 146   |16:34
itoperatorguy| free_ram_mb          | 10750 |16:34
itoperatorguy| local_gb             | 156   |16:34
itoperatorguy| local_gb_used        | 10    |16:34
itoperatorguy| memory_mb            | 12286 |16:34
itoperatorguy| memory_mb_used       | 1536  |16:34
itoperatorguy| running_vms          | 1     |16:34
itoperatorguy| vcpus                | 4     |16:34
itoperatorguy| vcpus_used           | 1     |16:34
itoperatorguy+----------------------+-------+16:34
itoperatorguyroot@controller1 (adminv3) $> openstack flavor show magnum.test16:35
itoperatorguy+----------------------------+--------------------------------------+16:35
itoperatorguy| Field                      | Value                                |16:35
itoperatorguy+----------------------------+--------------------------------------+16:35
itoperatorguy| OS-FLV-DISABLED:disabled   | False                                |16:35
itoperatorguy| OS-FLV-EXT-DATA:ephemeral  | 0                                    |16:35
itoperatorguy| access_project_ids         | None                                 |16:35
itoperatorguy| disk                       | 10                                   |16:35
itoperatorguy| id                         | 4c883f16-f2bd-4653-9729-40140a81f73c |16:35
itoperatorguy| name                       | magnum.test                          |16:35
itoperatorguy| os-flavor-access:is_public | True                                 |16:35
itoperatorguy| properties                 |                                      |16:35
itoperatorguy| ram                        | 512                                  |16:35
itoperatorguy| rxtx_factor                | 1.0                                  |16:35
itoperatorguy| swap                       |                                      |16:35
itoperatorguy| vcpus                      | 1                                    |16:35
*** rcernin has quit IRC16:35
yatinkarelitoperatorguy, looks OK16:36
yatinkarelitoperatorguy, when you tried with fedora-atomic-latest did you used the same flavor magnum.test16:37
itoperatorguyI still can scroll back to the nova console-log output from the previously failed (stucked) cluster16:37
fishbonestrigazi: I am currently trying the fedora-atomic-newton image and the docker daemon maintains appropriately, however; I cannot establish any remote connections to the cluster16:38
itoperatorguyhttp://pastebin.com/9Kz4ifej16:39
yatinkarelitoperatorguy, share log over paste.openstack.org16:39
*** tonanhngo has joined #openstack-containers16:39
fishboneIf i use the fedora-atomic-latest image I can establish a connection, but it produces the error that there are no healthy nodes, and on the nodes the docker daemon does not restart properly16:39
itoperatorguyfishbone: I had the same experience with fedora-atomic-latest16:39
fishboneitoperatorguy: Is there something I may be screwing up?16:40
itoperatorguyI am here to find out that for myself as well :D16:40
fishboneit says that the host is actively refusing the connection16:40
fishboneoh perfect, at least I am not alone in this16:41
itoperatorguysame was here. Now trying with fedora-atomic-newton instead of latest but this time cluster creation is just hanging in the CREATING_IN_PROGRESS state16:41
itoperatorguyyatinkarel I copied to pastebin http://pastebin.com/9Kz4ifej16:42
itoperatorguyshould I share on paste.openstack.org as well?16:42
yatinkarelitoperatorguy, its Ok on pastebin16:42
fishboneitoperatorguy: hmm, are you sure the fedora-atomic-newton images created in the cluster can reach your magnum api controller by IP16:42
itoperatorguyhttp://paste.openstack.org/show/599440/16:43
yatinkarelitoperatorguy, i think i have seen this error sometimes, retrying creating a cluster solved this16:43
fishboneitoperatorguy: that was an issue I suffered from previously, I modified the host file in the image to be able to route to the controller16:43
itoperatorguyfishbone: I don't think the newton image even booting properly until the point when it is trying to do that16:44
*** Oku_OS is now known as Oku_OS-away16:45
itoperatorguylatest image I think was able. At least when run tcpdump I saw that packages are coming back and forth16:45
yatinkarelitoperatorguy, what's the status for test-vm16:45
itoperatorguyroot@controller1 (adminv3) $> openstack server list16:46
itoperatorguy+--------------------------------------+-------------------------------------------------------+--------+---------------------------------------------------------------------------------+----------------------+16:46
itoperatorguy| ID                                   | Name                                                  | Status | Networks                                                                        | Image Name           |16:46
itoperatorguy+--------------------------------------+-------------------------------------------------------+--------+---------------------------------------------------------------------------------+----------------------+16:46
*** vijaykc4 has joined #openstack-containers16:46
itoperatorguy| ff4bedba-075d-45bd-97fc-57762b01e929 | test-fedora                                           | ERROR  |                                                                                 |                      |16:46
itoperatorguy| eec4bc1a-76a6-4618-9d7d-9d6612688896 | test-vm                                               | ERROR  |                                                                                 | fedora-atomic-newton |16:46
itoperatorguy| 4434246e-1490-45c5-b731-5dfca5b9be6a | sw-24rpy4pic-0-6xpgfhkgdozi-swarm_master-zjcfshuqjfd5 | ACTIVE | swarm-cluster-5ne2unwbaxn3-fixed_network-rphluerivq4v=10.0.0.6, 192.168.100.118 |                      |16:46
itoperatorguy+--------------------------------------+-------------------------------------------------------+--------+---------------------------------------------------------------------------------+----------------------+16:46
itoperatorguytest-fedore: created via dashboard error message:  Failed to perform requested operation on instance "test-fedora", the instance has an error status: Please try again later [Error: Build of instance ff4bedba-075d-45bd-97fc-57762b01e929 aborted: Block Device Mapping is Invalid.].16:46
yatinkarelitoperatorguy, what flavor you used when you created cluster-template16:47
itoperatorguytest-vm created visa 'openstack cli' error message:16:47
itoperatorguy fault                                | {u'message': u'No valid host was found. There are not enough hosts available.', u'code': 500, u'details': u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 496, in       |16:47
itoperatorguy|                                      | build_instances\n    context, request_spec, filter_properties)\n  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 567, in _schedule_instances\n    hosts =                  |16:47
itoperatorguy|                                      | self.scheduler_client.select_destinations(context, spec_obj)\n  File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 370, in wrapped\n    return func(*args, **kwargs)\n  File     |16:47
itoperatorguy|                                      | "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 51, in select_destinations\n    return self.queryclient.select_destinations(context, spec_obj)\n  File                   |16:47
yatinkarelAre you able to login to 192.168.100.118?16:47
itoperatorguy|                                      | "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 37, in __run_method\n    return getattr(self.instance, __name)(*args, **kwargs)\n  File "/usr/lib/python2.7/site-        |16:47
itoperatorguy|                                      | packages/nova/scheduler/client/query.py", line 32, in select_destinations\n    return self.scheduler_rpcapi.select_destinations(context, spec_obj)\n  File "/usr/lib/python2.7/site-                |16:47
itoperatorguy|                                      | packages/nova/scheduler/rpcapi.py", line 126, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/usr/lib/python2.7/site-                            |16:48
itoperatorguy|                                      | packages/oslo_messaging/rpc/client.py", line 169, in call\n    retry=self.retry)\n  File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 97, in _send\n    timeout=timeout,    |16:48
itoperatorguy|                                      | retry=retry)\n  File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send\n    retry=retry)\n  File "/usr/lib/python2.7/site-                                |16:48
itoperatorguy|                                      | packages/oslo_messaging/_drivers/amqpdriver.py", line 455, in _send\n    raise result\n', u'created': u'2017-02-17T16:30:59Z'}                                                                      |16:48
itoperatorguy| flavor                               | magnum.test (4c883f16-f2bd-4653-9729-40140a81f73c)16:48
itoperatorguyno I can't16:48
itoperatorguyflavor is costum flavor callled magnum.test16:48
itoperatorguyroot@controller1 (adminv3) $> openstack flavor show magnum.test16:48
itoperatorguy<itoperatorguy> +----------------------------+--------------------------------------+16:48
itoperatorguy<itoperatorguy> | Field                      | Value                                |16:48
itoperatorguy<itoperatorguy> +----------------------------+--------------------------------------+16:48
itoperatorguy<itoperatorguy> | OS-FLV-DISABLED:disabled   | False                                |16:48
itoperatorguy<itoperatorguy> | OS-FLV-EXT-DATA:ephemeral  | 0                                    |16:48
yatinkarelitoperatorguy, please try with some larger flavor16:48
itoperatorguy<itoperatorguy> | access_project_ids         | None                                 |16:49
itoperatorguy<itoperatorguy> | disk                       | 10                                   |16:49
itoperatorguy<itoperatorguy> | id                         | 4c883f16-f2bd-4653-9729-40140a81f73c |16:49
itoperatorguy<itoperatorguy> | name                       | magnum.test                          |16:49
itoperatorguy<itoperatorguy> | os-flavor-access:is_public | True                                 |16:49
itoperatorguy<itoperatorguy> | properties                 |                                      |16:49
itoperatorguy<itoperatorguy> | ram                        | 512                                  |16:49
itoperatorguy<itoperatorguy> | rxtx_factor                | 1.0                                  |16:49
itoperatorguy<itoperatorguy> | swap                       |                                      |16:49
itoperatorguy<itoperatorguy> | vcpus                      | 1                                    |16:49
itoperatorguy* rcernin has quit (Quit: WeeChat 1.6)16:49
yatinkareli use m1.small(2096MB ram, 20 GB, 1cpu)16:49
*** pcastets has quit IRC16:49
yatinkareltry with larger flavor16:49
itoperatorguyok I will try it16:50
itoperatorguyI know 'it depends' but how long should a cluster creation take with only 1 node?16:53
*** faizy has quit IRC16:54
*** fragatina has quit IRC16:54
*** salmankhan has quit IRC16:57
*** salmankhan has joined #openstack-containers16:58
*** yatinkarel has quit IRC16:58
fishbonehello all, despite which flavor size I use for the cluster template in the fedora-atomic-newton image provisioning of the cluster I am unable to make a docker connection16:59
*** vijaykc4 has quit IRC16:59
fishboneI receive the error, The target machine actively refused it17:00
fishboneI am able to ssh and ping all of the nodes in the cluster, and I can run docker commands from within all of the nodes17:00
fishbonewhen using the fedora-atomic-latest image I can establish a connection but the docker daemon on the nodes consistently fails, and I receive the error "There are no healthy nodes available in the cluster"17:02
*** EricGonczer_ has quit IRC17:05
*** jberkus has joined #openstack-containers17:05
*** vijaykc4 has joined #openstack-containers17:06
itoperatorguycluster is still CREATE_IN_PROGRESS state with fedora-atomic-newton17:06
itoperatorguyI see that a new instance come up, but can not ping, ssh or anything with the machine + I can not interact with it in the dashboard web-console17:06
itoperatorguylike is still booting...17:07
itoperatorguyand this is only the swarm_master instance, there is now 'slave' instance at all17:08
*** EricGonczer_ has joined #openstack-containers17:08
*** yatinkarel has joined #openstack-containers17:10
*** ttx has left #openstack-containers17:10
*** salmankhan has quit IRC17:12
itoperatorguyI had to go, but thanks everybody for your time17:12
*** itoperatorguy has quit IRC17:13
*** jberkus has quit IRC17:13
*** salmankhan has joined #openstack-containers17:16
*** vijaykc4 has quit IRC17:19
*** jberkus has joined #openstack-containers17:19
*** Serlex has quit IRC17:25
fishbonethis is the message I see in the docker daemon on the nodes, http://paste.openstack.org/show/599442/17:27
fishboneI'm wondering if it's an issue with the ssl cert creation?17:27
*** vijaykc4 has joined #openstack-containers17:29
*** dimtruck is now known as zz_dimtruck17:33
yatinkarelfishbone, ls /etc/docker17:39
fishboneyatinkarel, I will try that asap, I tried reconstructing the cluster as I received an error when running that command manually, http://paste.openstack.org/show/599445/17:41
fishboneyatinkarel: ....led me to believe the cluster settings were the issue17:41
fishboneyatinkarel: http://paste.openstack.org/show/599445/, my bad17:42
*** vijaykc4 has quit IRC17:45
yatinkarelfishbone, when you create the cluster, swarm-master starts and swarm-agent doesn't?17:45
*** zz_dimtruck is now known as dimtruck17:46
*** jerrygb_ has joined #openstack-containers17:46
fishboneyatinkarel, they both start fine initially it seems that may have been a hiccup, when I restart the docker daemon on both it restart fine, however; I was still not able to connect through a docker client17:46
*** vimal has quit IRC17:47
fishboneyatarinkel: scratch that, same error, although the service does restart fine17:48
fishboneyatinkarel: contents of /etc/docker, ca.crt  conf  server.crt  server.csr  server.key17:48
yatinkarelfishbone, certs are correct17:49
yatinkarelfishbone, docker service not running?17:49
fishboneyatinkarel: not initially, although I can restart it17:50
fishboneyatinkarel: I received the same error from http://paste.openstack.org/show/599442/17:50
*** jerrygb has joined #openstack-containers17:50
fishboneyatinkarel: and when manually entering the failed command, same thing, http://paste.openstack.org/show/599445/17:51
*** jerrygb_ has quit IRC17:51
fishboneyatinkarel: actually something new, http://paste.openstack.org/show/599446/17:52
yatinkarelfishbone, just: sudo pkill docker17:53
fishboneyatinkarel: and this time I can't restart the service....17:53
yatinkarelfishbone, sudo rm -rf /var/lib/docker17:54
yatinkarelsudo systemctl start docker17:54
fishbonerm: cannot remove ‘/var/lib/docker/devicemapper’: Device or resource busy17:54
yatinkarelps -eaf|grep docker17:55
fishbonefedora    1766  1278  0 17:55 pts/0    00:00:00 grep --color=auto docker17:55
fishboneyatinkarel: the process continues to respawn17:56
yatinkarelcluster-template-create command?17:57
yatinkarelcat /etc/redhat-release17:57
fishboneFedora release 23 (Twenty Three)17:58
fishboneyatinkarel: this is on the swarm master instance17:58
yatinkarelfishbone, ok17:58
yatinkarelfishbone, can you share cluster-template-create command used by you17:59
fishboneyatinkarel: I did it in the dashboard but I imagine it was simply, openstack cluster create --cluster-template test test18:00
fishboneIt auto set the 1 master 1 node amounts18:00
yatinkarelnot cluster-create, cluster-template-create18:00
yatinkarelmagnum cluster-template-list18:01
fishboneyatinkarel: oh sorry, those settings are here, http://paste.openstack.org/show/599447/18:01
yatinkarelfishbone, cinder list18:02
*** pbandark has joined #openstack-containers18:02
fishboneyatinkarel, also the image-id is fedora-atomic-latest but it is actually the fedora-atomic-newton image, I had an issue where if I used a different name it would fail cluster creation18:03
yatinkarel:( really18:03
fishboneyatinkarel: here, http://paste.openstack.org/show/599448/, thanks for your help18:04
fishboneyatarinkel: oh yes, although I think that issue has subsided but I would rather not edit the image since it currently works18:07
*** sdake_ has joined #openstack-containers18:10
*** sdake_ has quit IRC18:10
*** sdake_ has joined #openstack-containers18:10
yatinkarelfishbone, can you try with overlay storage driver18:11
yatinkarelmagnum cluster-template-create --name swarm-cluster-template-overlay \18:11
yatinkarel                       --image-id fedora-atomic-latest \18:11
yatinkarel                       --keypair-id INFRASTRUCTURE \18:11
yatinkarel                       --external-network-id Internet \18:11
yatinkarel                       --dns-nameserver 8.8.8.8 \18:11
yatinkarel                       --flavor m1.small \18:11
yatinkarel                       --coe swarm --docker-storage-driver overlay18:11
yatinkarelmagnum cluster-create --cluster-template swarm-cluster-template-overlay18:12
fishboneyatinkarel: I will try that18:12
fishboneyatinkarel: thank you18:12
*** randallburt has quit IRC18:13
*** sdake has quit IRC18:14
*** fragatina has joined #openstack-containers18:17
fishboneyatinkarel: OMGSSS I can't believe it was that, you are a life savor.... Any idea why the devicemapper driver is not working?18:23
*** dfflanders has quit IRC18:31
*** EricGonc_ has joined #openstack-containers18:32
Drago1fishbone: Woohoo!18:34
*** EricGonczer_ has quit IRC18:34
yatinkarelfishbone, No idea right now, but will try to reproduce this on monday18:43
*** jberkus has quit IRC18:52
*** jberkus has joined #openstack-containers18:56
fishboneyatinkarel, thanks again18:58
-openstackstatus- NOTICE: Restarting gerrit due to performance problems19:02
*** sdake_ has quit IRC19:10
*** salmankhan has quit IRC19:20
*** fragatina has quit IRC19:47
*** ssbarnea has quit IRC19:50
*** lakerzhou_ has joined #openstack-containers20:18
*** xiangxinyong has joined #openstack-containers20:23
*** edisonxiang has quit IRC20:25
lakerzhou_Hi There. I am new to Magnum. Trying setup magnum with devstack by following Quick Start. But creation of k8s-cluster failed after about 30 minutes. Heat logs show kube_masters CREATE_FAILED. My Ubuntu 14.04 VM has 32G RAM and 6 vcpu. Any advice?20:26
*** salmankhan has joined #openstack-containers20:30
openstackgerritOpenStack Release Bot proposed openstack/magnum master: Update reno for stable/ocata  https://review.openstack.org/43559320:32
*** salmankhan has quit IRC20:34
*** amoralej is now known as amoralej|off20:37
*** jerrygb has quit IRC20:44
*** mtanino has quit IRC20:51
*** fragatina has joined #openstack-containers20:55
*** pbandark has quit IRC21:00
swatson_lakerzhou_: I've had problems with Devstacks on VMs21:08
swatson_I forgot the root cause but it had to do with VM support for some feature that I can't remember now :(21:08
*** salmankhan has joined #openstack-containers21:17
*** dimtruck is now known as zz_dimtruck21:22
*** Jeffrey4l_ has joined #openstack-containers21:34
*** Jeffrey4l has quit IRC21:34
*** tonanhngo has quit IRC21:38
*** fragatin_ has joined #openstack-containers21:38
*** chris_hultin is now known as chris_hultin|AWA21:38
lakerzhou_swatson_: thanks for the response. I will try passthrough cpumode, can try again21:40
*** fragatina has quit IRC21:40
*** Drago1 has quit IRC21:52
*** jerrygb has joined #openstack-containers21:57
*** jerrygb has quit IRC22:03
*** tonanhngo has joined #openstack-containers22:16
*** fragatin_ has quit IRC22:16
*** dave-mccowan has quit IRC22:17
*** fragatina has joined #openstack-containers22:17
*** lakerzhou_ has quit IRC22:17
*** Serlex has joined #openstack-containers22:30
*** sdake has joined #openstack-containers22:30
*** code34 has joined #openstack-containers22:37
*** jwcroppe has quit IRC22:40
*** zz_dimtruck is now known as dimtruck22:49
openstackgerritVijendar Komalla proposed openstack/magnum master: (WIP)Validate project-id on quota create/update  https://review.openstack.org/43562322:57
*** jerrygb has joined #openstack-containers23:00
*** Serlex has quit IRC23:00
*** jwcroppe has joined #openstack-containers23:05
*** jwcroppe has quit IRC23:09
*** catintheroof has quit IRC23:21
*** catintheroof has joined #openstack-containers23:22
*** swatson_ has quit IRC23:24
*** catintheroof has quit IRC23:26
*** salmankhan has quit IRC23:30
*** salmankhan has joined #openstack-containers23:37
*** PagliaccisCloud has quit IRC23:46

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!