Today, our game server project encountered a problem, the server can’t not handle request from client.
then I start debugging server. I found that server still can receive HTTP post from client, but each time server received the request, it will be blocked after below line code:
var grain = GrainClient.GrainFactory.GetGrain<IMatchMakingParticipantGrain>(param.PlayerId);
Look like GetGrain() method has been blocked after I have invoked it.
Then I thought it maybe is the reason the Grain have not receive the Promises from a very previous GetGrain() invoke.
Orleans official website have publish a document, the title of that document is <Orleans: Distributed Virtual Actors for Programmability and Scalability>, the author is “Philip A. Bernstein, Sergey Bykov, Alan Geller, Gabriel Kliot, Jorgen ThelinMicrosoft Research”, that document is public, you can search it if you have interesting in it. In the document I found the 2.4 phase about “Promises“, it introduce Promises about Orleans like below:
Actors interact by sending asynchronous messages. As
in most modern distributed systems programming
models, these message exchanges are exposed as method
calls. However, unlike traditional models, Orleans
method calls return immediately with a promise for a
future result, rather than blocking until the result is
returned. Promises allow for concurrency without
requiring explicit thread management.
Promises have a three-state lifecycle. Initially, a
promise is unresolved; it represents the expectation of
receiving a result at some future time. When the result is
received, the promise becomes fulfilled and the result
becomes the value of the promise. If an error occurs,
either in the calculation of the result or in the
communication, the promise becomes broken.
After found above information in that document, I finally confirmed the reason we have been blocked when invoke method GetGrain() is because some code inside function grain.InvokeGrainMethod() accidentally invoke GrainFactory.GetGrain() with the same grain id again, and if you invoke GetGrain() with the same grain id inside your grain task, it will cause the first time grain.InvokeGrainMethod() can not got promises. so it will blocked in GetGrain() next time invoked.
here is the wrong process cause block:
after found the reason block the server, I fixed it by removing the GetGrain() code inside InvokeGrainMethod(), then the problem resolved.
so never try to GetGrain() with the same grain id inside a Grain task. it will cause block acident.