[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: imageChange build trigger's and the internal registry



We probably should be using the upstream Dockerfile parsing, even if it doesn't make it trivial.


> On Feb 19, 2015, at 9:01 AM, Ben Parees <bparees redhat com> wrote:
> 
> 
> ----- Original Message -----
>> From: "Brenton Leanhardt" <bleanhar redhat com>
>> To: "Ben Parees" <bparees redhat com>
>> Cc: "Openshift Dev" <dev lists openshift redhat com>
>> Sent: Thursday, February 19, 2015 8:47:33 AM
>> Subject: Re: imageChange build trigger's and the internal registry
>> 
>> +++ Brenton Leanhardt [19/02/15 07:47 -0500]:
>>> +++ Ben Parees [18/02/15 18:47 -0500]:
>>>> 
>>>> ----- Original Message -----
>>>>> From: "Brenton Leanhardt" <bleanhar redhat com>
>>>>> To: "Ben Parees" <bparees redhat com>
>>>>> Cc: "Openshift Dev" <dev lists openshift redhat com>
>>>>> Sent: Wednesday, February 18, 2015 6:42:46 PM
>>>>> Subject: Re: imageChange build trigger's and the internal registry
>>>>> 
>>>>> +++ Ben Parees [18/02/15 18:13 -0500]:
>>>>>> 
>>>>>> ----- Original Message -----
>>>>>>> From: "Brenton Leanhardt" <bleanhar redhat com>
>>>>>>> To: "Ben Parees" <bparees redhat com>
>>>>>>> Cc: "Openshift Dev" <dev lists openshift redhat com>
>>>>>>> Sent: Wednesday, February 18, 2015 5:54:01 PM
>>>>>>> Subject: Re: imageChange build trigger's and the internal registry
>>>>>>> 
>>>>>>> +++ Ben Parees [18/02/15 16:43 -0500]:
>>>>>>>> 
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Brenton Leanhardt" <bleanhar redhat com>
>>>>>>>>> To: "Ben Parees" <bparees redhat com>
>>>>>>>>> Cc: "Openshift Dev" <dev lists openshift redhat com>
>>>>>>>>> Sent: Wednesday, February 18, 2015 4:32:13 PM
>>>>>>>>> Subject: Re: imageChange build trigger's and the internal registry
>>>>>>>>> 
>>>>>>>>> +++ Brenton Leanhardt [18/02/15 16:22 -0500]:
>>>>>>>>>> +++ Ben Parees [18/02/15 16:01 -0500]:
>>>>>>>>>>> The lack of a BaseImage on your BuildConfig is part(all?) of the
>>>>>>>>>>> problem.
>>>>>>>>>>> the BaseImage field on a DockerBuildStrategy has to match to the
>>>>>>>>>>> "image"
>>>>>>>>>>> field on the image change trigger, and will be replaced with the
>>>>>>>>>>> pull
>>>>>>>>>>> spec
>>>>>>>>>>> computed from the imageRepo.
>>>>>>>>>> 
>>>>>>>>>> I've attached my build config.  I _think_ I have the BaseImage
>>>>>>>>>> wired
>>>>>>>>>> up correctly but it's still not working.
>>>>>>>>> 
>>>>>>>>> I see the FROM line being replaced with the BaseImage:
>>>>>>>>> https://github.com/openshift/origin/blob/master/pkg/build/builder/docker.go#L157
>>>>>>>>> 
>>>>>>>>> How is it supposed to know what registry to pull from though?
>>>>>>>> 
>>>>>>>> that's all supposed to be ImageRepository magic...   by default it
>>>>>>>> assumes
>>>>>>>> the local docker-registry service i believe.  Otherwise you can
>>>>>>>> specify
>>>>>>>> the
>>>>>>>> repository in the ImageRepo definition.
>>>>>>> 
>>>>>>> I looked through the ImageChangeController code and I confirmed that
>>>>>>> the build is being created correctly:
>>>>>>> 
>>>>>>> 17:23:44.456478   55989 generate.go:34] Substituting
>>>>>>> 172.30.17.215:5001/ose3/ose-base:211c76c5ba1506641425d5bfd56adb12f672cbf7203bf33b9ea5560b06f0d028
>>>>>>> for ose3/ose-base
>>>>>>> 
>>>>>>> I can even 'docker pull' that image on my Node.  I'm confused I'm not
>>>>>>> seeing that image url in my build log when it outputs the FROM line.
>>>>>>> I'll keep digging in to this tomorrow.
>>>>>> 
>>>>>> when you get a chance run "osc get build -n <namespace> <buildid> -o
>>>>>> json"
>>>>>> 
>>>>>> you should see the BaseImage field containing the substituted value, it
>>>>>> would be good to confirm.
>>>>> 
>>>>> Yeah, I checked that too.  It has the correct 172.30.17.215:5001 url
>>>>> in there.
>>>> 
>>>> hm, so the dockerfile isn't being rewritten correctly.  Any chance you have
>>>> an old origin-docker-builder image that might be missing that logic?
>> 
>> FWIW, I'm pretty sure this is due to the regex that is being used.
>> It's not taking multiline into account.  There are likely a few other
>> edge cases it doesn't handle like a lowercase FROM:
>> 
>> https://github.com/brenton/origin/commit/b17fad7d0d36a2e481a7cb1fbf519495931de455
>> 
>> I tested with that patch and my imagechange build trigger now works as
>> expected.
> 
> can you show me your dockerfile?  i don't understand why you'd need multiline matching.  
> 
>> 
>>> 
>>> As best I can tell my images are correct.  The docker-build image I'm
>>> running
>>> was build from the Origin v0.3.1 tag.  Here's some interesting details:
>>> 
>>> 1) You can see it's trying "FROM ose3/ose-base" which will obviously not
>>> work.
>>> It did however use /tmp/docker-build257371041/images/ose/Dockerfile.
>>> 
>>> # osc build-logs
>>> ddfe2250-b832-11e4-8ba2-fa163e2d6f31
>>> 2015-02-19T12:29:08.685088258Z Cloning into
>>> '/tmp/docker-build257371041'...
>>> 2015-02-19T12:29:48.575781232Z Switched to a new branch 'osebuild'
>>> 2015-02-19T12:29:48.575858910Z Branch osebuild set up to track remote
>>> branch osebuild from origin.
>>> 2015-02-19T12:29:48.577731042Z I0219 07:29:48.576760       1
>>> tar.go:133] Adding to tar:
>>> /tmp/docker-build257371041/images/ose/Dockerfile as Dockerfile
>>> 2015-02-19T12:29:48.577731042Z I0219 07:29:48.577027       1
>>> tar.go:133] Adding to tar:
>>> /tmp/docker-build257371041/images/ose/bin/.gitignore as bin/.gitignore
>>> 2015-02-19T12:29:48.669443494Z Step 0 : FROM ose3/ose-base
>>> 2015-02-19T12:29:48.669443494Z Pulling repository ose3/ose-base
>>> 2015-02-19T12:29:49.061959041Z 2015/02/19 07:29:49 Build error: Error:
>>> image ose3/ose-base not found
>>> 2015-02-19T12:32:00.613154279Z Cloning into
>>> '/tmp/docker-build339170468'...
>>> 
>>> 
>>> 2) If I check the json for the build I can see it has the correct baseImage.
>>> I also saw the corresponding log entry in journalctl:
>>> 
>>> Feb 19 07:28:53 bleanhar-v3-02-17-master.os1.phx2.redhat.com
>>> openshift[55989]:
>>> I0219 07:28:53.956463   55989 generate.go:34] Substituting
>>> 172.30.17.215:5001/ose3/ose-base:7880560543d799a0d9c4bf1e84a9fb869893d6edc1719bb9ea43af3fbe792756
>>> for ose3/ose-base
>>> 
>>> 
>>> # osc get -o json build ddfe2250-b832-11e4-8ba2-fa163e2d6f31
>>> {      "kind": "Build",
>>>  "apiVersion": "v1beta1",
>>>  "metadata": {
>>>      "name": "ddfe2250-b832-11e4-8ba2-fa163e2d6f31",
>>>      "namespace": "ose3",
>>>      "selfLink":
>>>      "/osapi/v1beta1/builds/ddfe2250-b832-11e4-8ba2-fa163e2d6f31?namespace=ose3",
>>>      "uid": "ddfe22b4-b832-11e4-8ba2-fa163e2d6f31",
>>>      "resourceVersion": "9702",
>>>      "creationTimestamp": "2015-02-19T07:28:54-05:00",
>>>      "labels": {
>>>          "buildconfig": "ose"
>>>      }
>>>  },
>>>  "parameters": {
>>>      "source": {
>>>          "type": "Git",
>>>          "git": {
>>>              "uri": "http://<scrubbed>/git/users/bleanhar/ose",
>>>              "ref": "osebuild"
>>>          }
>>>      },
>>>      "strategy": {
>>>          "type": "Docker",
>>>          "dockerStrategy": {
>>>              "contextDir": "images/ose",
>>>              "noCache": true,
>>>              "baseImage":
>>>              "172.30.17.215:5001/ose3/ose-base:7880560543d799a0d9c4bf1e84a9fb869893d6edc1719bb9ea43af3fbe792756"
>>>          }
>>>      },
>>>      "output": {
>>>          "to": {
>>>              "kind": "ImageRepository",
>>>              "name": "ose"
>>>          }
>>>      }
>>>  },
>>>  "status": "Failed",
>>>  "podName": "build-ddfe2250-b832-11e4-8ba2-fa163e2d6f31"
>>> 
>>> 3) If I cat out the Dockerfile from inside the image I can see OpenShift is
>>> indeed writing the ENV vars to it.  Somehow the FROM line isn't being
>>> replaced
>>> though:
>>> 
>>> -bash-4.2# cat /tmp/docker-build257371041/images/ose/Dockerfile
>>> #
>>> # This is the official OpenShift Origin image. It has as its entrypoint the
>>> OpenShift
>>> # all-in-one binary.
>>> #
>>> # The standard name for this image is openshift3_beta/ose
>>> #
>>> FROM ose3/ose-base
>>> 
>>> RUN yum install -y openshift && \
>>>  yum clean all
>>> 
>>> RUN ln -s /usr/bin/openshift /usr/bin/openshift-deploy && \
>>>  ln -s /usr/bin/openshift /usr/bin/openshift-experimental && \
>>>  ln -s /usr/bin/openshift /usr/bin/openshift-docker-build && \
>>>  ln -s /usr/bin/openshift /usr/bin/openshift-sti-build && \
>>>  ln -s /usr/bin/openshift /usr/bin/openshift-router
>>> 
>>> ENV HOME /root
>>> WORKDIR /var/lib/openshift
>>> ENTRYPOINT ["/usr/bin/openshift"]
>>> ENV OPENSHIFT_BUILD_REFERENCE osebuild
>>> ENV OPENSHIFT_BUILD_NAME ddfe2250-b832-11e4-8ba2-fa163e2d6f31
>>> ENV OPENSHIFT_BUILD_NAMESPACE ose3
>>> ENV OPENSHIFT_BUILD_SOURCE http://<scrubbed>/git/users/bleanhar/ose
>>> 
>>> 
>>> 
>>>> 
>>>> if you're in a position to do so, you might run hack/build-images.sh on
>>>> your host because we currently are working through some issues w/ not
>>>> always getting the right images pushed to dockerhub.  it's possible what's
>>>> out there isn't correct.
>>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Ben Parees | OpenShift
>>>>>>>>>>> 
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>> From: "Brenton Leanhardt" <bleanhar redhat com>
>>>>>>>>>>>> To: "Clayton Coleman" <ccoleman redhat com>
>>>>>>>>>>>> Cc: "Openshift Dev" <dev lists openshift redhat com>
>>>>>>>>>>>> Sent: Wednesday, February 18, 2015 3:35:56 PM
>>>>>>>>>>>> Subject: Re: imageChange build trigger's and the internal
>>>>>>>>>>>> registry
>>>>>>>>>>>> 
>>>>>>>>>>>> +++ Clayton Coleman [18/02/15 15:30 -0500]:
>>>>>>>>>>>>> Can you paste your build config?
>>>>>>>>>>>> 
>>>>>>>>>>>> It was mostly created with 'ex generate' and I made some minor
>>>>>>>>>>>> tweaks.
>>>>>>>>>>>> Below the only changes I made were to remove the git url
>>>>>>>>>>>> hostname:
>>>>>>>>>>>> 
>>>>>>>>>>>> # osc describe bc ose-base
>>>>>>>>>>>> Name:                   ose-base
>>>>>>>>>>>> Annotations:            <none>
>>>>>>>>>>>> Created:                2015-02-18 15:03:39 -0500 EST
>>>>>>>>>>>> Strategy:               Docker
>>>>>>>>>>>> No Cache:               yes
>>>>>>>>>>>> BaseImage:              <none>
>>>>>>>>>>>> Source Type:            Git
>>>>>>>>>>>> URL:
>>>>>>>>>>>> http://<snipped>/git/users/bleanhar/ose
>>>>>>>>>>>> Ref:                    osebuild
>>>>>>>>>>>> Output to:              ose-base
>>>>>>>>>>>> Output Spec:            <none>
>>>>>>>>>>>> Webhook Github:
>>>>>>>>>>>> localhost/osapi/v1beta1/buildConfigHooks/ose-base/DfI4Ka2eL5cwpZ-cus6A/github
>>>>>>>>>>>> Webhook Generic:
>>>>>>>>>>>> localhost/osapi/v1beta1/buildConfigHooks/ose-base/ydpu3GA0PGqFoG5a1Rqb/generic
>>>>>>>>>>>> 
>>>>>>>>>>>> # osc describe bc ose
>>>>>>>>>>>> Name:                           ose
>>>>>>>>>>>> Annotations:                    <none>
>>>>>>>>>>>> Created:                        2015-02-18 15:04:05 -0500 EST
>>>>>>>>>>>> Strategy:                       Docker
>>>>>>>>>>>> No Cache:                       yes
>>>>>>>>>>>> BaseImage:                      <none>
>>>>>>>>>>>> Source Type:                    Git
>>>>>>>>>>>> URL:
>>>>>>>>>>>> http://<snipped>/git/users/bleanhar/ose
>>>>>>>>>>>> Ref:                            osebuild
>>>>>>>>>>>> Output to:                      ose
>>>>>>>>>>>> Output Spec:                    <none>
>>>>>>>>>>>> Webhook Generic:
>>>>>>>>>>>> localhost/osapi/v1beta1/buildConfigHooks/ose/ydpu3GA0PGqFoG5a1Rqb/generic
>>>>>>>>>>>> Webhook Github:
>>>>>>>>>>>> localhost/osapi/v1beta1/buildConfigHooks/ose/DfI4Ka2eL5cwpZ-cus6A/github
>>>>>>>>>>>> Image Repository Trigger:       ose-base
>>>>>>>>>>>> - Tag:                          latest
>>>>>>>>>>>> - Image:                        ose3/ose-base
>>>>>>>>>>>> - LastTriggeredImageID:
>>>>>>>>>>>> 0975d5a7c29c921838fcc56dd154b94dc895b79c8417f720d818767652dbf779
>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Feb 18, 2015, at 2:45 PM, Brenton Leanhardt
>>>>>>>>>>>>>> <bleanhar redhat com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I have a working imageChange build trigger.  However, it seems
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>> the resulting build is trying to pull from the docker hub:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Build logs:
>>>>>>>>>>>>>> 2015-02-18T19:35:59.534994177Z Step 0 : FROM ose3/ose-base
>>>>>>>>>>>>>> 2015-02-18T19:35:59.534994177Z Pulling repository
>>>>>>>>>>>>>> ose3/ose-base
>>>>>>>>>>>>>> 2015-02-18T19:36:00.022455304Z 2015/02/18 14:36:00 Build
>>>>>>>>>>>>>> error:
>>>>>>>>>>>>>> Error:
>>>>>>>>>>>>>> image ose3/ose-base not found
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Curling my registry directly shows the image is there:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> curl
>>>>>>>>>>>>>> http://172.30.17.215:5001/v1/repositories/ose3/ose-base/tags/latest
>>>>>>>>>>>>>> "8a864801852bcee8d120b4fe815b78cb41ba3632ffaa0edd0dd213558464f60b
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> How is the resulting build supposed to find that image?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --Brenton
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> dev mailing list
>>>>>>>>>>>>>> dev lists openshift redhat com
>>>>>>>>>>>>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>>>>>>>>>>>> 
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> dev mailing list
>>>>>>>>>>>> dev lists openshift redhat com
>>>>>>>>>>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> dev mailing list
>>>>>>>>>> dev lists openshift redhat com
>>>>>>>>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>>> 
>>> _______________________________________________
>>> dev mailing list
>>> dev lists openshift redhat com
>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> 
> _______________________________________________
> dev mailing list
> dev lists openshift redhat com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]