Stream Data to VueJs Client Using .Net & gRPC
I decided to try gRPC, the name of which we heard a lot lately. But here, my aim was to try how it can be used instead of socketio etc. on the web, not among services as it is usually used. So I prepared a PoC for this, my goal was to test its feasibility and how to do it.
If you want to take a quick look at the project instead of reading how to do it, you can check it on github.
What & Why?
Before we begin, I want to give you a quick intro about gRPC and our development environment.
Let’s start with Remote Procedure Call (RPC), it’s not a new thing actually, in fact, it dates back to the 1970s. It is coded as if it were a normal (local) procedure call, without the programmer explicitly coding the details for the remote interaction. That is, the programmer writes essentially the same code whether the subroutine is local to the executing program, or remote.
So, then what is gRPC? gRPC was initially created by Google, which has used a single general-purpose RPC infrastructure called Stubby to connect the large number of microservices running within and across its data centers for over a decade. In March 2015, Google decided to build the next version of Stubby and make it open source. The result was gRPC, which is now used in many organizations outside of Google to power use cases from microservices to the “last mile” of computing (mobile, web, and Internet of Things). It uses protocol buffers as both its Interface Definition Language (IDL) and as its underlying message interchange format.
In our development environment we mainly use .net core for microservices and VueJs with typescript for our frontend. For a typical .net developer, we can say proto files are basically as interfaces and a bit of controllers on RestApis. But proto files need to be compiled to C#, ts or js code to work. For gRPC’s offical supported languages you can use protoc and for web you can use protoc-gen-grpc-web plugin. VS19 handles C# conversion internally with using grpc tools, so we only need to compile our proto file code to js.
Why I try to use gRPC?
In our IoT Platform; our customers are constantly checking their assets via reports or charts. But our microservices use REST, and none of them open to direct access. And, the efforts for serialization are wasted because of REST. Therefore, it makes sense to switch our microservices to grpc( We can keep REST too).
Main thing is; if we switch to grpc, will we still need signalr or socket.io to stream data to the web side?
TL;DR We can use gRPC instead of signalr or socket.io
Implementation
For this PoC, we are going to create a VueJs project and .Net 5 project into same solution. With this we can easily share same proto file. We use Envoy proxy for Http/2 to Http/1 conversion and vice versa. But for doing this and running service & the client at the same time we are going to dockerize our service and proxy.
Before we begin you need to install protoc (For compiling proto files)and grpc-web (For compilig proto files to js & d.ts files). There is a little problem with grpc-web and ts files so we only use d.ts and js files.
I’m gonna skip solution and project creation step. You need to create a “gRPC Service” and “Basic Vue.js Web Application”. Note that gRPC support comes only with Visual Studio 2019. Your solution should be something like that;
If you have any problems with project creation you can read microsoft docs.
Creating the Grpc Service
Create a folder named Protos and a proto file named “sensor.proto”
We are going to define our service, methods and messages etc. in this file. Like I said this is like defining an interface and combining it with a little bit of controller :) .
Let’s define our “Sensor” service and add a method named “ListenSensors”. If you look at the code below you’ll see our method returns stream.
While defining proto files we must obey some rules like not using same parameter id within the same object or zero based enums. For more information about proto syntax I highly recommend to read the docs.
If you use Visual Studio like me this file will be automatically compiled to C# code. Bu if you prefer to use other kind of ides you need to compile this file with protoc for to use.
Now we can implement our service. Let’s create a folder named “Services” and add a file named “SensorService.cs”. Your folder structure should something like this;
We’re going to use our compiled service. We define our service named as “Sensor” at the “sensor.proto” file so we can access it with that name. Let’s inspect our service code below.
As you can see, there is an abstract class named SensorBase which generated from our proto file definition. So we need to inherit from this class and implement our ListenSensors method. It’s a simple method which takes a request for multiple sensors and generates random value for each of the sensors and writes each result to stream. And if you notice there is a CancellationToken for stopping request, this can be called like request.cancel() at the client side when needed.
At this point we can create our docker file named “Dockerfile” for our service.
Proxy
At the transport layer gRPC uses HTTP/2 for request/response multiplexing by which client and servers can both initiate multiple streams on a single underlying TCP connection. We’re able to communicate between services or micro-services over HTTP/2 natively but not all browsers can use HTTP/2, some browsers still uses HTTP/1.1. At this point we wanted to stay at the safe point and decided to use a proxy to convert our packages from HTTP/1 to HTTP/2
For this we’re going to create a docker file named “envoy_Dockerfile” and “envoy.yaml” in the service’s directory(GrpcService/).
Our docker file is very basic, it thas docker image and config and runs. It should be something like below.
Before that we need to create our proxy configuration at the “envoy.yaml” file. At this point envoy’s documentation helps a lot, you can read it from here.
Firstly we need to create a listener which listens 8080, secondly this listener must filter request and route it to our service. So we need to define our service here as a cluster which named “sensor_service” and we say that, this service runs on this endpoint. At our example we’re deploy our proxy and service within same compose so cause of that endpoint address is “host.docker.internal” and service gonna run at port 9090
Now we can create our compose file. Let’s say Visual Studio to hey we can support docker. Right click to service project and within the add pane there should be Docker Support, you can read more information here.
Now you can go to docker compose file’s location and call those command on the command prompt
$ docker-compose build
$ docker-compose upCreating The Client
At the start we create a basic Vue.Js application and now we before we begin let’s talk about how can we use proto files here. With using grpc-web plugin we can compile our proto file to js. But at our company we use TypeScript with VueJs. So we need to convert our proto file to typescript but there is a problem. When I try to convert our proto file I saw mismatching definitions and ts file gives errors. This plugin says ts and d.ts generation is a experimental feature. At this point those errors give me chills and I’m about terminate PoC. But there is a miracle, compiling into only d.ts produces as I want. Let’s find out how we can.
For transform from proto to d.ts and js we need to call a protoc command every time when we change something at he our proto file. So I add a script named “proto” to package.json file for easy access.
On the first parameter we need to give our proto file’s path. second parameter defines our output path for js and d.ts files. Third parameter is the main thing; on this parameter we say “hey compile as js and d.ts” and with the last parameter we select which proto files we want to convert.
This creates a file like below for us;
As you can see, naming is changed a bit but this is acceptable. So we can use SensorClient or SensorPromiseClient they looks same :) .
Create a folder named “services” at the GrpcClient project and add a file named “sensorService.ts”. This is our dummy service, normally we can add authentication params etc here or can add extra business, but for now it can be dummy.
Your project structure should something like below;
Now lets use our service at the App.vue file.
First we import our definitions and we create a stream with our return type. And when user calls component’s listenSensors method we call our our service method and this returns a stream with our return type. When service writes someting to stream it’s gonna trigger our on “Data” handler. When user wants to stop it can call stopSensors method and within this method it calls stream.cancel(). This triggers CancellationToken at the service side.
Thanks to this PoC, I tried to help with gRPC and how to use it on the web. I hope I have been able to help you and convey what I know. If you want to inspect more, the PoC is on the Github, please take your time and inspect. Project offers insights about data rates or if you use HighCharts, how to render eighty thousand charts at the same time etc.
References
- https://grpc.io/about/#the-story-behind-grpc
- https://en.wikipedia.org/wiki/GRPC
- https://github.com/grpc/grpc
- https://www.envoyproxy.io/docs/envoy/latest/intro/what_is_envoy
- https://github.com/grpc/grpc-web
- https://grpc.io/docs/platforms/web/basics/
- https://docs.microsoft.com/tr-tr/aspnet/core/grpc/browser?view=aspnetcore-5.0
- https://dev.to/thearavind/a-todo-app-using-grpc-web-and-vuejs-3p55
