Not receiving data in a NestJS microservice - sockets

I'm trying to create a NestJS microservice that can listen for data on a specific port. Following the instructions from here, I have the following for main.ts:
import { NestFactory } from '#nestjs/core';
import { MicroserviceOptions, Transport } from '#nestjs/microservices';
import { AppModule } from './app.module';
const HOST = '127.0.0.1';
const PORT = 31123;
async function bootstrap() {
const app = await NestFactory.createMicroservice<MicroserviceOptions>(
AppModule,
{
transport: Transport.TCP,
options: {
host: HOST,
port: PORT
}
},
);
await app.listen();
}
bootstrap();
My app.controller.ts looks like:
import { Controller, Get } from '#nestjs/common';
import { MessagePattern } from '#nestjs/microservices';
import { AppService } from './app.service';
#Controller()
export class AppController {
constructor(private readonly appService: AppService) {}
#Get()
getHello(): string {
return this.appService.getHello();
}
#MessagePattern('hello')
dataHandler(data: string) {
console.log(`Received: ${data}`);
}
}
I'm attempting to send data to the microservice via a Python script, running on the same computer:
import socket
HOST = '127.0.0.1'
PORT = 31123
someData = 'hello'
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
print(f'Sending: "{someData}"')
encodedString = someData.encode('utf8')
s.sendall(bytearray(encodedString))
The data never gets through; the log message in the dataHandler function in the microservice never executes. Am I specifying the message-pattern incorrectly, or are NestJS microservices not designed to handle raw TCP socket data? I've read this answer and this answer, but those deal more with message broker situations. Any help would be greatly appreciated.

For anyone who lands here looking for a solution, I found the solution to this from this answer

Related

Nest JS user authentication issue with parameter name

I am just learning nestjs for about a day and I came across this strange bug, probably has something to do with me not understanding what Im doing and rushing the project so please bear with me. My main issue is that while using JWT authentication, JSON coming from body is "username" and I can't change it. I want to log in using {"email":"test#gmail.com", "password": "password123"}, but instead it only accepts {"username":"test#gmail.com", "password": "password123"}. The word "username" is not defined or mentioned anywhere in my codebase
users.controller.ts
import { Controller, Get, Post, Body, Param, UseGuards } from '#nestjs/common';
import { UsersService} from './users.service';
import { CreateUserDto} from './dto/create-user.dto';
import { AuthGuard} from '#nestjs/passport';
#Controller('/users')
export class UsersController {
// constructor(private readonly usersService: UsersService) {}
constructor(private readonly userService: UsersService) {}
#UseGuards(AuthGuard('jwt'))
#Get('username')
getUserByEmail(#Param() param) {
return this.userService.getUserByEmail(param.email);
}
#Post('register')
registerUser(#Body() createUserDto: CreateUserDto) {
return this.userService.registerUser(createUserDto);
}
}
users.service.ts
import { Injectable, BadRequestException } from '#nestjs/common';
import { CreateUserDto } from './dto/create-user.dto';
import { UpdateUserDto } from './dto/update-user.dto';
import { Model } from 'mongoose';
import { InjectModel } from '#nestjs/mongoose';
import { HashService } from './hash.service';
import { User, UserDocument} from '../schemas/user.schema'
#Injectable()
export class UsersService {
constructor(#InjectModel(User.name) private userModel: Model < UserDocument > , private hashService: HashService) {}
async getUserByEmail(email: string) {
return this.userModel.findOne({
email
})
.exec();
}
async registerUser(createUserDto: CreateUserDto) {
// validate DTO
const createUser = new this.userModel(createUserDto);
// check if user exists
const user = await this.getUserByEmail(createUser.email);
if (user) {
throw new BadRequestException();
}
// Hash Password
createUser.password = await this.hashService.hashPassword(createUser.password);
return createUser.save();
}
}
auth.controller.ts
import { AuthService} from './auth.service';
import { Controller, Request, UseGuards, Post} from '#nestjs/common';
import { AuthGuard } from '#nestjs/passport';
#Controller('auth')
export class AuthController {
constructor(private authService: AuthService) {}
#UseGuards(AuthGuard('local'))
#Post(`/login`)
async login(#Request() req) {
console.log(req.user, "here")
return this.authService.login(req.user);
}
}
Here is the source code https://github.com/networkdavit/pillicam_test
Any help or suggestion is highly appreciated!
I tried changing all the parameter names, user schemas, adding a DTO, I googled how to add a custom parameter name or override it, tried to find if "default username param" actually exists. Nothing has worked for me so far
It's in there username in your code. https://github.com/networkdavit/pillicam_test/blob/main/src/users/entities/user.entity.ts#:~:text=class%20User%20%7B-,username%3A%20string%3B,-password%3A%20string
You can change it.
Or you can refer to this article for JWT implementation in nest.js
Just in case anyone ever gets this problem, I found a solution.
All I had to do was to add this to my local.strategy.ts file in constructor
super({
usernameField: 'email',
passwordField: 'password'
});
The default expects a username and password, so have to modify it manually

Opening Mongoose connection in AdonisJS provider times out

I was following this article to use Mongo in AdonisJS 5 project.
I have an AdonisJS provider which I have created by node ace make:provider Mongo (it is registered in .adonisrc.json):
import { ApplicationContract } from '#ioc:Adonis/Core/Application'
import { Mongoose } from 'mongoose'
export default class MongoProvider {
constructor(protected app: ApplicationContract) {}
public async register() {
// Register your own bindings
const mongoose = new Mongoose()
// Connect the instance to DB
await mongoose.connect('mongodb://docker_mongo:27017/mydb')
// Attach it to IOC container as singleton
this.app.container.singleton('Mongoose', () => mongoose)
}
public async boot() {
// All bindings are ready, feel free to use them
}
public async ready() {
// App is ready
}
public async shutdown() {
// Cleanup, since app is going down
// Going to take the Mongoose singleton from container
// and call disconnect() on it
// which tells Mongoose to gracefully disconnect from MongoBD server
await this.app.container.use('Mongoose').disconnect()
}
}
My model is:
import { Schema, model } from '#ioc:Mongoose'
// Document interface
interface User {
email: string
}
// Schema
export default model(
'User',
new Schema<User>({
email: String,
})
)
Controller:
import { HttpContextContract } from '#ioc:Adonis/Core/HttpContext'
import User from 'App/Models/User'
export default class UsersController {
public async index({}: HttpContextContract) {
// Create a cat with random name
const cat = new User({
email: Math.random().toString(36).substring(7),
})
// Save cat to DB
await cat.save()
// Return list of all saved cats
const cats = await User.find()
// Return all the cats (including the new one)
return cats
}
}
And it is timeouting.
It is working, when I open the connection in controller like this though:
import { HttpContextContract } from '#ioc:Adonis/Core/HttpContext'
import User from 'App/Models/User'
import mongoose from 'mongoose'
export default class UsersController {
public async index({}: HttpContextContract) {
await mongoose.connect('mongodb://docker_mongo:27017/mydb')
// Create a cat with random name
const cat = new User({
email: Math.random().toString(36).substring(7),
})
// Save cat to DB
await cat.save()
// Return list of all saved cats
const cats = await User.find()
// Close the connection
await mongoose.connection.close()
// Return all the cats (including the new one)
return cats
}
}
I have just created an AdonisJS provider, registered it in .adonisrc.json, created a contracts/Mongoose.ts with typings, and use the model in controller.
Any idea? I'm stuck for a day with this.
Thanks
I managed to resolve this issue by not storing mongoose in a variable. It seems the mongoose variable you declare in your MongoProvider is the root of your timeout error.
So I did as follow :
export default class MongoProvider {
constructor(protected app: ApplicationContract) {}
public async register() {
await mongoose.connect('mongodb://localhost:27017/dbName')
this.app.container.singleton('Mongoose', () => mongoose)
}
public async boot() {
// All bindings are ready, feel free to use them
}
public async ready() {
// App is ready
}
public async shutdown() {
await this.app.container.use('Mongoose').disconnect()
}
}
If someone would be interested:
with the help of the article author the reason why it is not working was missing Mongoose when creating the model (Mongoose.model instead of just model:
export default Mongoose.model(
'User',
new Schema<User>({
email: String,
})
)
I followed this article too, and I have the same issue you discussed. but resolved this by importing mongoose in my model a little differently.
import mongoose in the model like this import Mongoose, { Schema } from '#ioc:Mongoose' instead of import { Schema, model } from '#ioc:Mongoose'
Example:
import Mongoose, { Schema } from '#ioc:Mongoose'
// Document interface
interface User {
email: string
}
// Schema
export default model(
'User',
new Schema<User>({
email: String,
})
)

Kafka with NestJS - send hangs

The code is pretty similar to the official tutorial: https://docs.nestjs.com/microservices/kafka
kafka.module.ts:
import { Module } from '#nestjs/common'
import { ClientsModule, Transport } from '#nestjs/microservices'
import { KafkaClientController } from './kafka.controller'
#Module({
imports: [
ClientsModule.register([
{
name: 'KAFKA_SERVICE',
transport: Transport.KAFKA,
options: {
client: {
clientId: 'say',
brokers: ['localhost:29092'], // docker container port
},
consumer: {
groupId: 'say-consumer',
},
},
},
]),
],
controllers: [KafkaClientController],
})
export class KafkaModule {}
kafka.controller.ts:
import { Controller, Get, Inject, OnModuleDestroy, OnModuleInit } from '#nestjs/common'
import { ClientKafka } from '#nestjs/microservices'
#Controller('kafka')
export class KafkaClientController implements OnModuleInit, OnModuleDestroy {
constructor(#Inject('KAFKA_SERVICE') private readonly kafka: ClientKafka) {}
async onModuleInit() {
this.kafka.subscribeToResponseOf('say.hello')
await this.kafka.connect()
}
onModuleDestroy() {
this.kafka.close()
}
#Get()
async sayHello() {
console.log('in sayHello')
const x = await this.kafka.send('say.hello', 'hi this is the data').toPromise() // hangs here
console.log(x)
}
}
The method sayHello hangs, even if I replace the send line with:
return this.kafka.send('say.hello', 'hi this is the data')
I've seen this question for Scala: Kafka producer hangs on send
But I couldn't find an ack option: KafkaOptions KafkaConfig ProducerConfig ConsumerConfig

Handle mongo/mongoose error in nestjs not

I would like to set up a filter to catch mongo errors (I'm using mongoose on this project) but nothing I do works and my research / test of what is on the internet does nothing.
mongoExceptionFilter.ts
import {
ArgumentsHost,
ConflictException,
BadRequestException,
Catch,
ExceptionFilter
} from '#nestjs/common';
import { MongoError } from 'mongodb';
#Catch(MongoError)
export class MongoExceptionFilter implements ExceptionFilter {
catch(exception: MongoError, host: ArgumentsHost): unknown {
switch (exception.code) {
case 11000: // duplicate exception
throw new ConflictException();
default:
throw new BadRequestException(`error ${exception.code}`);
}
}
}
I test a call here main.ts :
async function bootstrap() {
const app = await NestFactory.create(AppModule);
app.useGlobalPipes(new ValidationPipe());
app.useGlobalFilters(new MongoExceptionFilter());
await app.listen(3001);
}
and here users.controller.ts :
#Post()
#UseFilters(MongoExceptionFilter)
createUser(#Body() body: UsersDto): Promise<Users> {
return this.userService.createUser(body.name, body.password);
}
Some link i found just for information :
How to handle mongoose error with nestjs
NestJs - Catching MongoDB errors with a custom filter and #Catch(MongoError)
Same problem here. The only solution for me now is to catch the MongoError imported from mongodb in the global filter and check for it there
As per this MongoError from MongoDb is different from MongoError from mongoose package. So it seems you are using both the packages i.e. Mongodb and mongoose.
With Custom messages, I combined solutions from answers How to handle mongoose error with nestjs
import { ArgumentsHost, Catch, ExceptionFilter, RpcExceptionFilter } from '#nestjs/common';
import { Error } from 'mongoose';
import { IDTOError } from '../errors/bad-request-exception.error';
import ValidationError = Error.ValidationError;
import { MongoError } from 'mongodb';
#Catch(MongoError)
export class MongoExceptionFilter implements ExceptionFilter {
catch(exception: MongoError, host: ArgumentsHost) {
// switch (exception.code) {
// case 11000:
// default: console.log(exception,'ALERT ERROR CATCHED');
// // duplicate exception
// // do whatever you want here, for instance send error to client
// /** MAIGOD */
// }
const ctx = host.switchToHttp(),
response = ctx.getResponse();
return response.status(400).json(<IDTOError>{
statusCode: 400,
createdBy: 'ValidationErrorFilter, Schema or Model definition',
errors: exception,
});
}
}
#Catch(ValidationError)
export class ValidationErrorFilter implements RpcExceptionFilter {
catch(exception: ValidationError, host: ArgumentsHost): any {
const ctx = host.switchToHttp(),
response = ctx.getResponse();
return response.status(400).json(<IDTOError>{
statusCode: 400,
createdBy: 'ValidationErrorFilter, Schema or Model definition',
errors: exception.errors,
});
}
}

Expected 1 arguments when using storage in ionic

I tried using local storage in my ionic service provider app but I got this error.
[ts] Expected 1 arguments, but got 0. (alias) new Storage(config:
StorageConfig): Storage import Storage Create a new Storage instance
using the order of drivers and any additional config options to pass
to LocalForage.
Possible driver options are: ['sqlite', 'indexeddb', 'websql',
'localstorage'] and the default is that exact ordering.
Here is my source code:
import { Storage } from '#ionic/storage';
#Injectable()
export class MyServiceProvider {
public local : Storage;
mydata: any;
constructor(public http: Http) {
this.local = new Storage()
}
postLogin(data){
let link = "http://myapi.com/api/login.php";
return this.http.post(link, data)
.map(data => {
this.mydata = data;
console.log("data")
}, error =>{
console.log(error)
})
}
}
I didn't understand well what you want to do with the Storage, but according to Ionic Storage docs, you need to inject into your provider/component.
import { Storage } from '#ionic/storage';
export class MyApp {
constructor(private storage: Storage) { }
}
I suggest you to use that provider do get and set the data and you can call these methods wherever you import your provider.
public setData(nameData: string, valueData: any){
this.storage.set(nameData, valueData);
}
public getDataValue(nameData: string){
return this.storage.get(nameData);
}