Ocelot api gateway - kubernetes - error: "namespace:serviceservice:managementservice Unable to use ,it is invalid. Address must contain host only...." - kubernetes

The problem that i am facing is that the ocelot kubernetservicediscorverProvider does not seem to find the other services on the name space in kubernetes.My goal is to use api gateway to call apis in the other services in the same namespace. I currently get a http 404 Not Found error. And the api gateway pod, logs the following:
Ocelot.Provider.Kubernetes.KubernetesServiceDiscoveryProvider[0]
requestId: 0HM93C93DL2T0:00000003, previousRequestId: no previous request id, message: namespace:serviceservice:managementservice Unable to use ,it is invalid. Address must contain host only e.g. localhost and port must be greater than 0
warn: Ocelot.Responder.Middleware.ResponderMiddleware[0]
requestId: 0HM93C93DL2T0:00000003, previousRequestId: no previous request id, message: Error Code: ServicesAreEmptyError Message: There were no services in NoLoadBalancer errors found in ResponderMiddleware. Setting error response for request path:/api/management/User/3910, request method: GET
I suspect that i have mis configured something. I first tried using the Ocelot documentation, regarding kubernetes, but the documentation is out dated. (an example is the Type the sugest value does not work for more info go this github issue Docs/Kubernetes provider are wrong)
Then i went on searching online through github issues, stack overflow posts and even the source code. But i do not see have what i am lacking in my config.
I currently have kubernetes running localy, with minikube. The only things that i have seen online is that others have misconfigured the ocelot.json. But i do not see what i have done incorrectly in my config.
(Before trying ocelot on kubernetes i first try it with local hosts, to try out if it works and to see what it lacks. It apparantly lacked a middleware that could control jwt with different roles which had right to acces the end point. I have now written the middleware my self and it works on the local host config for ocelot)
My ocelot.json config file looks like this for kubernetes:
{
"Routes": [
{
"UpstreamPathTemplate": "/api/management/User/{everything}",
"UpstreamHttpMethod": [ "POST", "PUT", "GET" ],
"DownstreamPathTemplate": "/api/management/User/{everything}",
"DownstreamScheme": "http",
"ServiceName": "managementservice",
"AuthenticationOptions": {
"AuthenticationProviderKey": "Bearer",
"AllowedScopes": [ "CompanyId", "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier" ]
},
"RouteClaimsRequirement": { "role": "1,2,3" },
"AddHeadersToRequest": {
"CompanyId": "Claims[CompanyId] > value",
"UserId": "Claims[http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier] > value"
}
}
],
"GlobalConfiguration": {
"ServiceDiscoveryProvider": {
"Host": "127.0.0.1",
"Port": 8083,
"Namespace": "service",
"Type": "KubernetesServiceDiscoveryProvider"
}
}
}
my startup.cs ConfigureServices method looks like this
public void ConfigureServices(IServiceCollection services)
{
services.AddCors(c =>
{
c.AddPolicy("AllowOrigin", options => options.WithOrigins(Configuration["Cors:AllowOrigins"])
.AllowAnyHeader()
.AllowAnyMethod().AllowCredentials());
});
#region Authication settings
TokenValidationParameters tokenValidationParameters = new TokenValidationParameters
{
ValidateIssuerSigningKey = true,
IssuerSigningKey = new SymmetricSecurityKey(Encoding.ASCII.GetBytes(Configuration["Jwt:Key"])),
ValidateIssuer = false,
ValidateAudience = false,
ValidateLifetime = true,
ClockSkew = TimeSpan.Zero
};
services.AddSingleton(tokenValidationParameters);
services.AddAuthentication(
x =>
{
x.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
x.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;
}
)
.AddJwtBearer(JwtBearerDefaults.AuthenticationScheme, x =>
{
x.RequireHttpsMetadata = false;
x.SaveToken = true;
x.TokenValidationParameters = tokenValidationParameters;
});
#endregion
//Some more code
services.AddOcelot().AddKubernetes();
}
my startup.cs Configure method looks like this
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
// some more code
app.UseOcelot(configuration);
}
my program.cs CreateHostBuilder method looks like this
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.AddJsonFile("secrets/appsettings.kubernetes.json", optional: true)
.AddJsonFile("ocelot.json");
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});

It turns out that the real problem that i was having was in the permissions in kubernetes. The ocelot documentation also mentions this. But the command in the documentation is incorrect (most likely out dated).
This is the command that i used. Be warned the kubernetes documenation strongly disrecommend usage of this command permissive-rbac-permissions. But it is at least a way for you to test your api gateway in ocelot locally.
kubectl create clusterrolebinding permissive-binding --clusterrole=cluster-admin --user=admin --user=kubelet --group=system:serviceaccounts

Related

ASP.NET 6: Azure AD Authentication Infinite redirect loops with AWS Network LB and Fargate

I have a AWS Network Load balancer setup with a TLS (:443) Listener that forwards to a Target Group that is listening on port 8080.
The Target Group is an IP Type that points to a Fargate ECS instance.
My problem is that on that ECS instance my website is using Azure Ad for Auth. I got past the issue of the Redirect URI being HTTP instead of HTTPS, but now I am in a redirect loop that eventually ends in
We couldn't sign you in. Please try again.
I am using .NET 6 and Visual Studio 2022.
The Azure AD Auth was added via using the Connected Services in VS 2022.
The NLB URL has been added to Redirect URIs for the App in Azure AD.
Any help is appreciated.
appsettings.json
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*",
"AzureAd": {
"Instance": "https://login.microsoftonline.com/",
"Domain": "xxxxxxxxx.com",
"TenantId": "xxxxxxxxxx",
"ClientId": "xxxxxxxxxx",
"CallbackPath": "/signin-oidc"
},
"MicrosoftGraph": {
"BaseUrl": "https://graph.microsoft.com/v1.0",
"Scopes": "user.read"
}
}
program.cs
var builder = WebApplication.CreateBuilder(args);
var initialScopes = builder.Configuration["MicrosoftGraph:Scopes"]?.Split(' ');
builder.Services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
.AddMicrosoftIdentityWebApp(builder.Configuration.GetSection("AzureAd"))
.EnableTokenAcquisitionToCallDownstreamApi(initialScopes)
.AddMicrosoftGraph(builder.Configuration.GetSection("MicrosoftGraph"))
.AddInMemoryTokenCaches();
builder.Services.AddAuthorization(options =>
{
options.FallbackPolicy = options.DefaultPolicy;
});
// Add services to the container.
builder.Services.AddRazorPages().AddMicrosoftIdentityUI();
builder.Services.AddScoped<IDynamoDBConnection, DynamoDBConnection>();
builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
options.KnownNetworks.Clear();
options.KnownProxies.Clear();
});
builder.WebHost.UseUrls("http://*:8080");
var app = builder.Build();
//This is what fixes the Http redirect URI issue. Problem is it causes a redirect loop
app.Use((context, next) =>
{
context.Request.Scheme = "https";
return next(); //return next(context); //rewritten 8/19 8:23 no change
});
app.UseForwardedHeaders();
// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Error");
// The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
app.MapRazorPages();
app.MapControllers();
app.Run();
I have tried multiple browsers and the issue is the same.
I ran into this same issue and managed to resolve by adding the client secret to the appsettings.json.
In the Azure portal, go to Active Directory -> App registrations -> your-app -> Certificates & secrets. Add a new client secret, copy the Value (not the Secret ID, I gave myself an extra headache making that mistake) and paste it into your appsettings Azure object like so:
"AzureAd": {
"Instance": "https://login.microsoftonline.com/",
"Domain": "xxxxxxxxx.com",
"TenantId": "xxxxxxxxxx",
"ClientId": "xxxxxxxxxx",
"CallbackPath": "/signin-oidc",
"ClientSecret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}

How to use Ionic proxy in conjunction with AWS SDK

Using Ionic 4.4.0 and aws-sdk 2.157.0. I'm trying to create an S3 bucket from my local web browser, but am running into CORS problems when attempting to run the following code, method createBucketByCompanyKey():
import { Injectable } from '#angular/core';
import * as AWS from 'aws-sdk';
#Injectable()
export class AwsProvider {
private accessKeyId:string = 'myAccessKey';
private secretAccessKey:string = 'mySuperSecret';
private region:string = 'us-east-1';
constructor() {
AWS.config.update({accessKeyId: this.accessKeyId, secretAccessKey: this.secretAccessKey, region: this.region});
}
createBucketByCompanyKey(companyKey){
let s3 = new AWS.S3();
let params = {
Bucket: companyKey,
CreateBucketConfiguration: {
LocationConstraint: this.region
}
};
s3.createBucket(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
}
}
This gives me the error
Failed to load https://s3.amazonaws.com/-KwzdjmyrHiMBCqHH1ZC: Response
to preflight request doesn't pass access control check: No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://localhost:8100' is therefore not allowed
access. The response had HTTP status code 403.
Which led me to this post here after several hours of googling. It appears I need to run ionic through a proxy. I've also tried changing my "path" to http://localhost:8100, but stuck I remain.
{
"name": "MyApp",
"app_id": "",
"type": "ionic-angular",
"integrations": {},
"proxies": [
{
"path": "/",
"proxyUrl": "https://s3.amazonaws.com/"
}
]
}
I've also come across posts telling my to download a Chrome extension that disables CORS, but that didn't work either.
Any ideas on how to setup this proxy to work with AWS' SDK?
Forget the proxies. For Mac, enter in the following in the terminal to open a Google Chrome browser with CORS disabled.
open -a Google\ Chrome --args --disable-web-security --user-data-dir
Compliments of this post.

Why is IdentityServer redirecting to http rather than https?

I have a very simple MVC5 website that I'm trying to secure with IdentityServer3.
Both my website and my IdentityServer instance are hosted as separate sites in AppHarbor. Both are behind https.
When I hit a resource in my website that is protected by an [Authorize] attribute (e.g., /Home/About), I am successfully redirected to IdentityServer, and I can successfully authenticate.
When IdentityServer POSTs its response back to the website (via app.FormPostResponse.js), the website responds with a 302 redirect to the requested resource - as expected. However, this redirect is to http, not https (see the network trace below).
I'm sure this is just something wrong with my IdentityServer config, but I'd appreciate any pointers as to what I've got wrong.
(AppHarbor uses a reverse proxy (nginx I believe) in front of IIS, where SSL terminates - so I have RequireSsl = false for this scenario, as per the IdentityServer documentation.)
Here is my website's Startup.cs
public class Startup
{
public void Configuration(IAppBuilder app)
{
app.UseCookieAuthentication(new CookieAuthenticationOptions
{
AuthenticationType = "Cookies"
});
app.UseOpenIdConnectAuthentication(new OpenIdConnectAuthenticationOptions
{
Authority = "https://<my-idsrv3>.apphb.com/identity",
ClientId = "<my-client-id>",
Scope = "openid profile roles email",
RedirectUri = "https://<my-website>.apphb.com",
ResponseType = "id_token",
SignInAsAuthenticationType = "Cookies",
UseTokenLifetime = false
});
JwtSecurityTokenHandler.InboundClaimTypeMap = new Dictionary<string, string>();
}
}
Here is Startup.cs from my IdentityServer3 instance:
public class Startup
{
public void Configuration(IAppBuilder app)
{
app.Map("/identity", idsrvApp =>
{
idsrvApp.UseIdentityServer(new IdentityServerOptions
{
SiteName = "My Identity Server",
SigningCertificate = Certificates.LoadSigningCertificate(),
RequireSsl = false,
PublicOrigin = "https://<my-idsrv3>.apphb.com",
Factory = new IdentityServerServiceFactory()
.UseInMemoryUsers(Users.Get())
.UseInMemoryClients(Clients.Get())
.UseInMemoryScopes(Scopes.Get())
});
});
}
}
Here is the definition of my website Client:
new Client
{
Enabled = true,
ClientName = "My Website Client",
ClientId = "<my-client-id>",
Flow = Flows.Implicit,
RedirectUris = new List<string>
{
"https://<my-website>.apphb.com"
},
AllowAccessToAllScopes = true
}
Here is the trace from Chrome, after clicking 'Yes, Allow' on the IdentityServer consent screen:
So it looks like this issue was caused by my client website being behind an SSL-terminating nginx front-end.
With reference to this GitHub issue, I added the following to the start of my website's app configuration:
app.Use(async (ctx, next) =>
{
string proto = ctx.Request.Headers.Get("X-Forwarded-Proto");
if (!string.IsNullOrEmpty(proto))
{
ctx.Request.Scheme = proto;
}
await next();
});
This makes the website aware that incoming requests were over https; this in turn appears to ensure that the IdentityServer3 middleware generates https uri's.
Had the same issue running identityserver4 in an Azure App Service. Even with forced https, the generated urls in .well-known/openid-configuration were still http://.
Fixed using the same solution as the other answer, but using AspNetCore ForwardedHeadersExtensions:
var forwardOptions = new ForwardedHeadersOptions
{
ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto,
// Needed because of mixing http and https.
RequireHeaderSymmetry = false,
};
// Accept X-Forwarded-* headers from all sources.
forwardOptions.KnownNetworks.Clear();
forwardOptions.KnownProxies.Clear();
app.UseForwardedHeaders(forwardOptions);
See also https://github.com/IdentityServer/IdentityServer4/issues/1331 for more discussion on this subject.
Add forwarded headers in your startup
services.Configure<ForwardedHeadersOptions>(options =>
{
options.ForwardedHeaders =
ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto | ForwardedHeaders.XForwardedHost;
});
and
app.UseForwardedHeaders(new ForwardedHeadersOptions()
{
ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
});
Finally tell the config it has to replace the http to https in the redirect url. I'm still looking for a better way to implement this.
in your .addopenidconnect() add:
Func<RedirectContext, Task> redirectToIdentityProvider = (ctx) =>
{
if (!ctx.ProtocolMessage.RedirectUri.StartsWith("https") && !ctx.ProtocolMessage.RedirectUri.Contains("localhost"))
ctx.ProtocolMessage.RedirectUri = ctx.ProtocolMessage.RedirectUri.Replace("http", "https");
return Task.FromResult(0);
};
opt.Events = new OpenIdConnectEvents
{
OnRedirectToIdentityProvider = redirectToIdentityProvider
};

Reset Local Admin User on VM using AzureRM Rest api

I'm using the AzureRM rest api to communicate to the hypervisor. One of the things I need to do is to reset the local admin password on a VM, however I can't figure out how to reset it.
We could use the Virtual Machine Extensions REST API to do that. It works correctly for me. Following is my detail test info.
1.We need to get the authorization in the request header
Authorization:Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUz.......
Content-Type:application/json
2.Add the following info in the request body
{
"properties": {
"publisher": "Microsoft.Compute",
"type": "VMAccessAgent",
"typeHandlerVersion": "2.0",
"autoUpgradeMinorVersion": true,
"settings": {
"UserName": "local admin" //your local admin
},
"protectedSettings": {
"Password": "your reset passord" //match the password policy
}
},
"location": "East Asia"
}
Send the http request with Fiddler.
4 . Successfully remote the VM with reset password.
We also can reset our local admin password in the Azure portal.
You can also use the Azure .NET SDK, which has a wrapper for the create or update call.
I tried and ran into an issue where the VM extension request went through fine and was installed successfully, but the password was not updated.
After attaching fiddler, I saw that my dynamic objects for settings and protectedSettings were not getting properly serialized by the library. The solution was to pass a dictionary down to the VirtualMachineExtensions() constructor instead.
Before:
proxy.VirtualMachineExtensions.BeginCreateOrUpdateWithHttpMessagesAsync(
"<resource group>",
"<vm name>",
"<you name it>",
new Microsoft.Azure.Management.Compute.Models.VirtualMachineExtension(
location: "westus",
publisher: "Microsoft.Compute",
virtualMachineExtensionType: "VMAccessAgent",
typeHandlerVersion: "2.0",
autoUpgradeMinorVersion: true,
settings: new
{
UserName: "<username>"
},
protectedSettings: new
{
Password: "<password>"
}));
After:
proxy.VirtualMachineExtensions.BeginCreateOrUpdateWithHttpMessagesAsync(
"<resource group>",
"<vm name>",
"<you name it>",
new Microsoft.Azure.Management.Compute.Models.VirtualMachineExtension(
location: "westus",
publisher: "Microsoft.Compute",
virtualMachineExtensionType: "VMAccessAgent",
typeHandlerVersion: "2.0",
autoUpgradeMinorVersion: true,
settings: new Dictionary<string, string>()
{
{ "UserName", "<username>" }
},
protectedSettings: new Dictionary<string, string>()
{
{"PassWord", "<password>" }
}));

Using Grunt to Mock Endpoints

I'm using Yeoman, Grunt, and Bower, to construct a platform for building a frontend independently of a a backend. The idea would be that all of my (AngularJS) controller, services, factories, etc live in this project, and get injected afterwards into my serverside codebase based off the result of grunt build.
My question is:
How can I mock endpoints so that the Grunt server responds to the same endpoints as my (Rails) App will?
At the moment I am using:
angular.module('myApp', ['ngResource'])
.run(['$rootScope', function ($rootScope) {
$rootScope.testState = 'test';
}]);
And then in each of my individual services:
mockJSON = {'foo': 'myMockJSON'}
And on every method:
if($rootScope.testState == 'test'){
return mockJSON;
}
else {
real service logic with $q/$http goes here
}
Then after grunt build, testState = 'test' gets removed.
This is clearly a relatively janky architecture. How can I avoid it? How can I have Grunt respond to the same endpoints as my app (some of which have dynamic params) apply some logic (if necessary), and serve out a json file (possibly dependent on path params)?
I've fixed this issue by using express to write a server that responds with static json.
First I created a directory in my project called 'api'. Within that directory I have the following files:
package.json:
{
"name": "mockAPI",
"version": "0.0.0",
"dependencies": {
"express": "~3.3.4"
}
}
Then I run npm install in this directory.
index.js:
module.exports = require('./lib/server');
lib/server.js:
express = require('express');
var app = express();
app.get('/my/endpoint', function(req, res){
res.json({'foo': 'myMockJSON'});
});
module.exports = app
and finally in my global Gruntfile.js:
connect: {
options: {
port: 9000,
hostname: 'localhost',
},
livereload: {
options: {
middleware: function (connect, options) {
return [
lrSnippet,
mountFolder(connect, '.tmp'),
mountFolder(connect, yeomanConfig.app),
require('./api')
];
}
}
},
Then the services make the requests, and the express server serves the correct JSON.
After grunt build, the express server is simply replaced by a rails server.
As of grunt-contrib-connect v.0.7.0 you can also just add your custom middleware to the existing middleware stack without having to manually rebuild the existing middleware stack.
livereload: {
options: {
open: true,
base: [
'.tmp',
'<%= config.app %>'
],
middleware: function(connect, options, middlewares) {
// inject a custom middleware into the array of default middlewares
middlewares.push(function(req, res, next) {
if (req.url !== '/my/endpoint') {
return next();
}
res.writeHead(200, {'Content-Type': 'application/json' });
res.end("{'foo': 'myMockJSON'}");
});
return middlewares;
}
}
},
See https://github.com/gruntjs/grunt-contrib-connect#middleware for the official documentation.
Alternatively you can use the grunt-connect-proxy to proxy everything that is missing in your test server to an actual backend.
It's quite easy to install, just one thing to remember when adding proxy to your livereload connect middleware is to add it last, like this:
middleware: function (connect) {
return [
lrSnippet,
mountFolder(connect, '.tmp'),
mountFolder(connect, yeomanConfig.app),
proxySnippet
];
}
grunt-connect-prism is similar to the Ruby project VCR. It provides an easy way for front end developers to record HTTP responses returned by their API (or some other remote source) and replay them later. It's basically an HTTP cache, but for developers working on a Single Page Application (SPA). You can also generate stubs for API calls that don't exist, and populate them the way you want.
It's useful for mocking complex & high latency API calls during development. It's also useful when writing e2e tests for your SPA only, removing the server from the equation. This results in much faster execution of your e2e test suite.
Prism works by adding a custom connect middleware to the connect server provided by the grunt-contrib-connect plugin. While in 'record' mode it will generate a file per response on the filesystem with content like the following:
{
"requestUrl": "/api/ponies",
"contentType": "application/json",
"statusCode": 200,
"data": {
"text": "my little ponies"
}
}
DISCLAIMER: I'm the author of this project.
You can use Apache proxy and connect your REST server with gruntjs.
Apache would do this:
proxy / -> gruntjs
proxy /service -> REST server
you would use your application hitting Apache and angular.js application would think that is talking with itself so no cross domain problem.
Here is a great tutorial on how to set this up:
http://alfrescoblog.com/2014/06/14/angular-js-activiti-webapp-with-activiti-rest/
Just my alternative way that based on Abraham P's answer. It does not need to install express within 'api' folder. I can separate the mock services for certain files. For example, my 'api' folder contains 3 files:
api\
index.js // assign all the "modules" and then simply require that.
user.js // all mocking for user
product.js // all mocking for product
file user.js
var user = function(req, res, next) {
if (req.method === 'POST' && req.url.indexOf('/user') === 0) {
res.end(
JSON.stringify({
'id' : '5463c277-87c4-4f1d-8f95-7d895304de12',
'role' : 'admin'
})
);
}
else {
next();
}
}
module.exports = user;
file product.js
var product = function(req, res, next) {
if (req.method === 'POST' && req.url.indexOf('/product') === 0) {
res.end(
JSON.stringify({
'id' : '5463c277-87c4-4f1d-8f95-7d895304de12',
'name' : 'test',
'category': 'test'
})
);
}
else {
next();
}
}
module.exports = product;
index.js just assigns all the "modules" and we simply require that.
module.exports = {
product: require('./product.js'),
user: require('./user.js')
};
My Gruntfile.js file
connect: {
options: {
port: 9000,
// Change this to '0.0.0.0' to access the server from outside.
hostname: 'localhost',
livereload: 35729
},
livereload: {
options: {
open: true,
middleware: function (connect) {
return [
connect.static('.tmp'),
connect().use(
'/bower_components',
connect.static('./bower_components')
),
connect.static(appConfig.app),
require('./api').user,
require('./api').product,
];
}
}
}