Custom json converter with freezed without a wrapper class - flutter

Question

I'm using the freezed package to work with immutable models and make use of the built-in feature for json serialization by the json_serializable package. I have a simple User class/model with different union types (UserLoggedIn, UserGeneral, UserError):
#freezed
class User with _$User {
const factory User(String id, String email, String displayName) =
UserLoggedIn;
const factory User.general(String email, String displayName) = UserGeneral;
const factory User.error(String message) = UserError;
factory User.fromJson(Map<String, dynamic> json) => _$UserFromJson(json);
}
Since I'm using multiple constructors and don't want my API to include the runtimeType key as suggested by the documentation, I can write a converter (scroll a bit more down, sentence starts with: If you don't control the JSON response, then you can implement a custom converter.).
So based on that I wrote the following converter class:
class UserConverter implements JsonConverter<User, Map<String, dynamic>> {
const UserConverter();
#override
User fromJson(Map<String, dynamic> json) {
if (json['id'] != null) {
return UserLoggedIn.fromJson(json);
} else if (json['error'] != null) {
return UserError.fromJson(json);
} else {
return UserGeneral.fromJson(json);
}
}
#override
Map<String, dynamic> toJson(User data) => data.toJson();
}
The documentation now references another class (a wrapper class) which would now use this converter via annotation, something like this:
#freezed
class UserModel with _$UserModel {
const factory UserModel(#UserConverter() User user) = UserModelData;
factory UserModel.fromJson(Map<String, dynamic> json) =>
_$UserModelFromJson(json);
}
Question: is it possible to make use of this converter without having to use a wrapper class (UserModel)?
Reasoning: this wrapper class is adding another layer of abstraction which is not needed (in my cases). Especially since the wrapper class does not have any other benefit / purpose and it feels like it should be possible to do that without using it.