Fleet Allocation questions

Tom Rota

unread,

Sep 4, 2018, 11:28:57 AM9/4/18

to agones-discuss

Hi,

Could anyone tell me what I'm doing wrong while trying to allocate a gameserver? I'd really appreciate the help. The environment is Agones from agones-install-0.4.0.rc.zip on GKE installed with Helm.

There are some more questions inline below.

These are imported:

import (
    "net/http"
    "agones.dev/agones/pkg/client/clientset/versioned"
    "agones.dev/agones/pkg/apis/stable/v1alpha1"
    "agones.dev/agones/pkg/util/runtime" // for the logger
    "github.com/gin-gonic/gin"
    corev1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/client-go/rest"
)

Constants for namespace, fleetname, and generatename:

const namespace = "default"

const fleetname = "somefleet"

const generatename = "somefleet-"

Is this the correct way to use the FleetAllocationsGetter?

// get a FleetAllocationInterface for the namespace default

fleetAllocationInterface := agonesClient.StableV1alpha1().FleetAllocations(namespace)

Is this the right way to define the fleet allocation?

// define the fleet allocation

        fa := &v1alpha1.FleetAllocation{
            ObjectMeta: metav1.ObjectMeta{
                GenerateName: generatename, Namespace: namespace,
            },
                Spec: v1alpha1.FleetAllocationSpec{FleetName: fleetname},
            }

The call to Create raises "error: "Internal error occurred: Internal error occurred: jsonpatch replace operation does not apply: doc is missing key: /status/GameServer". Any idea why?

        // make a new fleet allocation
        newFleetAllocation, err := fleetAllocationInterface.Create(fa)
        if err != nil {
            logger.WithError(err).Fatal("Failed to create fleet allocation for ", fleetname)
        } else {
            logger.Info("Created a fleet allocation for ", fleetname)

}

Using kubectl create and a yaml file, fleet allocation works as expected.

Thanks in advance!

Tom

Mark Mandel

unread,

Sep 4, 2018, 11:34:19 AM9/4/18

to Tom Rota, agones-discuss

That is strange - not seen that one before.

A few things we can look at:

1) Can you kubectl describe the FleetAllocation that you have created with the api?

2) Can you send through the logs from the controller when you try and allocate (kubectl logs --namespace=agones-system agones-<tabcomplete> - we should be able to see an issue there

Mark

--
You received this message because you are subscribed to the Google Groups "agones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to agones-discus...@googlegroups.com.
To post to this group, send email to agones-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/agones-discuss/df3e5d3f-b7d3-4dd9-abae-4c0d3d572db9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Tom Rota

unread,

Sep 4, 2018, 12:09:48 PM9/4/18

to agones-discuss

Here's a gameserver that was allocated through the api. I don't have a FleetAllocation resource...should there be?

Controller logs coming up.

$ kubectl describe gs pymuws-clfzv-q7q5g

Name:         pymuws-clfzv-q7q5g
Namespace:    default
Labels:       stable.agones.dev/gameserverset=pymuws-clfzv
Annotations: <none>
API Version: stable.agones.dev/v1alpha1
Kind:         GameServer
Metadata:
Cluster Name:
Creation Timestamp: 2018-09-04T15:43:22Z
Finalizers:
    stable.agones.dev
Generate Name: pymuws-clfzv-
Generation:     1
Owner References:
    API Version:           stable.agones.dev/v1alpha1
    Block Owner Deletion: true
    Controller:            true
    Kind:                  GameServerSet
    Name:                  pymuws-clfzv
    UID:                   40c9d095-b059-11e8-95e0-42010a8e00c1
Resource Version:        2912
Self Link:               /apis/stable.agones.dev/v1alpha1/namespaces/default/gameservers/pymuws-clfzv-q7q5g
UID:                     40d1c258-b059-11e8-95e0-42010a8e00c1
Spec:
Container: pymuws-server
Health:
    Failure Threshold:      3
    Initial Delay Seconds: 60
    Period Seconds:         5
Ports:
    Container Port: 26000
    Host Port:       7777
    Name:            default
    Port Policy:     static
    Protocol:        UDP
Template:
    Metadata:
      Creation Timestamp: <nil>
    Spec:
      Containers:
        Image:              index.docker.io/trota/pymuws
        Image Pull Policy: Always
        Name:               pymuws-server
        Resources:
      Image Pull Secrets:
        Name: regcred
Status:
Address:    104.196.62.83
Node Name: gke-agones-pymuws-cluste-default-pool-a6421e45-93jd
Ports:
    Name: default
    Port: 7777
State:   Allocated
Events:
Type    Reason     Age   From                        Message
----    ------     ---- ----                        -------
Normal Creating   12m   gameserver-controller       Pod pymuws-clfzv-q7q5g-qfxlx created
Normal Starting   12m   gameserver-controller       Address and Port populated
Normal Ready      9m    gameserver-controller       SDK.Ready() executed
Normal Allocated 55s   fleetallocation-controller Allocated from Fleet pymuws

Tom Rota

unread,

Sep 4, 2018, 12:13:12 PM9/4/18

to agones-discuss

Here is output from kubectl logs --namespace=agones-system agones-controller-66457c6f54-47gvc

after trying to allocate:

{"level":"info","msg":"Processing obj","obj":"default/pymuws-clfzv","queue":"stable.agones.dev.GameServerSetController","source":"*gameserversets.Controller","time":"2018-09-04T16:06:27Z"}

On Tuesday, September 4, 2018 at 11:34:19 AM UTC-4, Mark Mandel wrote:

Mark Mandel

unread,

Sep 4, 2018, 3:51:13 PM9/4/18

to Tom Rota, agones-discuss

Hmn, so no errors.

But the gameserver you showed, you did successfully allocate it? Or did some other thing happen, not entirely sure I follow? How did it get to an allocated state?

As a potentially silly question - is the fleet you are trying to allocate from, does it exist? :)

Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/agones-discuss/79bbddb6-7dac-481b-ae3b-f47aa610eab3%40googlegroups.com.

Tom Rota

unread,

Sep 4, 2018, 4:30:29 PM9/4/18

to agones-discuss

Yeah, that is the weird part...how did it get to an allocated state?

The error from the pod running the service is error: "Internal error occurred: Internal error occurred: jsonpatch replace operation does not apply: doc is missing key: /status/GameServer" sounds like it comes from a package that gets imported by Agones. At least, I couldn't find any error strings like that in the repo.

Yes, the fleet exists. I check for that and log it. I was wondering the same thing, as it seems like it gets lost. If it knows the fleet, it should know the gameserver spec, right?

Here is the service I'm writing if you wouldn't mind looking.

https://github.com/slartibaartfast/agones/tree/allocate/examples/service

Maybe I am going about this the wrong way?

Mark Mandel

unread,

Sep 4, 2018, 4:50:53 PM9/4/18

to Tom Rota, agones-discuss

Hmnn. I am confused as well. It seems like it should be fine. Sounds like K8s doesn't like the JSONPatch generated here:

https://github.com/GoogleCloudPlatform/agones/blob/master/pkg/fleetallocation/controller.go#L168

Sounds like we also need an e2e test for this.

What K8s version/platform are you on?

Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/agones-discuss/29bb6d4e-234d-455c-8685-5d1486af1c6a%40googlegroups.com.

Tom Rota

unread,

Sep 5, 2018, 9:54:07 AM9/5/18

to agones-discuss

My kubectl version...

Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.7", GitCommit:"dd5e1a2978fd0b97d9b78e1564398aeea7e7fe92", GitTreeState:"clean", BuildDate:"2018-04-19T00:05:56Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.6-gke.2", GitCommit:"384b4eaa132ca9a295fcb3e5dfc74062b257e7df", GitTreeState:"clean", BuildDate:"2018-08-15T00:10:14Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

I'll check the values being passed to jsonpatch.CreatePatch.

Mark Mandel

unread,

Sep 5, 2018, 11:13:45 AM9/5/18

to Tom Rota, agones-discuss

I'm going to also jump on writing an e2e test for this, see what i can find as well.

To view this discussion on the web visit https://groups.google.com/d/msgid/agones-discuss/fde45caa-b128-47d3-b2b8-0352e17dd40e%40googlegroups.com.

Tom Rota

unread,

Sep 5, 2018, 11:57:11 AM9/5/18

to agones-discuss

That would be great. I see you guys have some issues logged for more e2e tests.

I found some errors in my agones-system agones-controller logs. These occur after bringing up the fleet and prior to calling Create(fa). Maybe something is just misconfigred?

First this...

A little later on, maybe when the gs calls /ready, it seems to forget the controller service account...

ERROR: logging before flag.Parse: E0905 15:02:13.198532 1 event.go:200] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pymuws-jfcqh-9tdtv.155189236c4e7322", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"440", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"GameServer", Namespace:"default", Name:"pymuws-jfcqh-9tdtv", UID:"518f9f95-b11c-11e8-92af-42010a8e00e5", APIVersion:"stable.agones.dev", ResourceVersion:"1374", FieldPath:""}, Reason:"Ready", Message:"SDK.Ready() executed", Source:v1.EventSource{Component:"gameserver-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63671756533, loc:(*time.Location)(0x1bc3a60)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbedc19dd4a68fb41, ext:158982060102, loc:(*time.Location)(0x1bc3a60)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "pymuws-jfcqh-9tdtv.155189236c4e7322" is forbidden: User "system:serviceaccount:agones-system:agones-controller" cannot patch events in the namespace "default": Unknown user "system:serviceaccount:agones-system:agones-controller"' (will not retry!)

Mark Mandel

unread,

Sep 6, 2018, 11:38:57 PM9/6/18

to Tom Rota, agones-discuss

You'll be pleased to know, I created an e2e test:

https://github.com/markmandel/agones/blob/e2e/fleets/test/e2e/fleet_test.go#L25

And got exactly the same error.

So I don't think you are doing anything wrong -- this is a very strange one. Digging in!

Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/agones-discuss/e10237eb-a3ff-4527-a02b-e93b72a9a65d%40googlegroups.com.

Mark Mandel

unread,

Sep 7, 2018, 1:29:33 AM9/7/18

to Tom Rota, agones-discuss

Hi!

So I have a fix here (and the PR shows what the issue is)

https://github.com/GoogleCloudPlatform/agones/pull/347

I'd love some user testing - can you see if the client from this PR will solve your problem, or if you have to do a full deploy with the build from this PR to solve it?

It may also be possible to create a workaround, by setting the values for the Gameserver status and other nullable pointers.

Considering doing a hotfix to solve this issue.

Thanks for finding this bug! There is now an e2e test to make sure it never happens again!

Mark

Tom Rota

unread,

Sep 7, 2018, 10:33:33 AM9/7/18

to agones-discuss

Thanks Mark! The fix you describe in the PR makes sense.

I'm installing it now on gke with agones.image.tag=0.5.0-03f4866 and will let you know how it goes.

Tom Rota

unread,

Sep 7, 2018, 11:04:19 AM9/7/18

to agones-discuss

Your fix works great! Now

fleetAllocationInterface := agonesClient.StableV1alpha1().FleetAllocations(namespace)

newFleetAllocation, err := fleetAllocationInterface.Create(fa)

creates one allocation, and no errors.

Good to know the end to end test is keeping an eye on the future.

A hotfix might not be neccessary. This test was successful after downloading the agones-install-0.4.0.zip and then

helm install --name pymuws-server --namespace agones-system agones --set agones.image.tag=0.5.0-03f4866

Also, thank you for the workaround idea.

Mark Mandel

unread,

Sep 7, 2018, 11:20:51 AM9/7/18

to Tom Rota, agones-discuss

Excellent - glad to hear it's working for you!

MArk

To view this discussion on the web visit https://groups.google.com/d/msgid/agones-discuss/677561ae-7dff-413f-9c68-7e2513bbc5c6%40googlegroups.com.

Reply all

Reply to author

Forward