i am using these conditions in ha proxy
use_backend test if { path_beg -i /test/ } { { ssl_fc_has_crt } || { src 10.0.0.25 } || { src 10.1.0.152 } || { src 10.0.2.41 } || { src 10.0.0.158} || { src 64.32.12.252 } || { src 35.43.19.101 } || { src 80.240.254.1 } || { src 82.10.80.7 } }
But i am seeing this error:
error detected while parsing switching rule : missing fetch method in ACL expression '{'.
Docs show { } like it could group ACLs, but doesn't elaborate on it:
A condition is formed as a disjunctive form:
[!]acl1 [!]acl2 ... [!]acln { or [!]acl1 [!]acl2 ... [!]acln } ...
So maybe it isn't grouping at all. All examples with {} i can see are for anonymous ACLs.
You want to combine AND with OR in one condition and haproxy isn't very helpful, but here is something that should work:
acl allowed_to_test_site src 10.0.0.25 10.0.0.24 10.1.0.152 10.0.2.41 10.0.0.158 64.32.12.252 35.43.19.101 80.240.254.1 82.10.80.7
acl allowed_to_test_site ssl_fc_has_crt
use_backend backend-sonar if { path_beg -i /test/ } allowed_to_test_site
Few points to explain here:
src IP1 || src IP2 can be declared as acl ip src IP1 IP2 and so on. such list works as multiple OR
declaring ACL multiple times works again as multiple OR
Docs say:
acl <aclname> <criterion> [flags] [operator] <value> ...
Declare or complete an access list.
which could use some more explicit explaination
3. AND is implicit
This way we get the logic of (path_beg -i /test/) AND ( ssl_fc_has_crt OR src matches one of the IPs)
Maybe one day haproxy will have better syntax for that.
Related
I have an ansible host file that is stored in GitHub and was wondering if there is a way to list out all the host in jenkins with choice parameters? Right now every time I update the host file in Github I would have to manually go into each Jenkins job and update the choice parameter manually. Thanks!
I'm assuming your host file has content something similar to below.
[client-app]
client-app-preprod-01.aws-xxxx
client-app-preprod-02.aws
client-app-preprod-03.aws
client-app-preprod-04.aws
[server-app]
server-app-preprod-01.aws
server-app-preprod-02.aws
server-app-preprod-03.aws
server-app-preprod-04.aws
Option 01
You can do something like the one below. Here you can first checkout the repo and then ask for the user input. I have implemented the function getHostList() to parse the host file to filter the host entries.
pipeline {
agent any
stages {
stage('Build') {
steps {
git 'https://github.com/jglick/simple-maven-project-with-tests.git'
script {
def selectedHost = input message: 'Please select the host', ok: 'Next',
parameters: [
choice(name: 'PRODUCT', choices: getHostList("client-app","ansible/host/location"), description: 'Please select the host')]
echo "Host:::: $selectedHost"
}
}
}
}
}
def getHostList(def appName, def filePath) {
def hosts = []
def content = readFile(file: filePath)
def startCollect = false
for(def line : content.split('\n')) {
if(line.contains("["+ appName +"]")){ // This is a starting point of host entries
startCollect = true
continue
} else if(startCollect) {
if(!line.allWhitespace && !line.contains('[')){
hosts.add(line.trim())
} else {
break
}
}
}
return hosts
}
Option 2
If you want to do this without checking out the source and with Job Parameters. You can do something like the one below using the Active Choice Parameter plugin. If your repository is private, you need to figure out a way to generate an access token to access the Raw GitHub link.
properties([
parameters([
[$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
description: 'Select the Host',
name: 'Host',
script: [
$class: 'GroovyScript',
fallbackScript: [
classpath: [],
sandbox: false,
script:
'return [\'Could not get Host\']'
],
script: [
classpath: [],
sandbox: false,
script:
'''
def appName = "client-app"
def content = new URL ("https://raw.githubusercontent.com/xxx/sample/main/testdir/hosts").getText()
def hosts = []
def startCollect = false
for(def line : content.split("\\n")) {
if(line.contains("["+ appName +"]")){ // This is a starting point of host entries
startCollect = true
continue
} else if(startCollect) {
if(!line.allWhitespace && !line.contains("[")){
hosts.add(line.trim())
} else {
break
}
}
}
return hosts
'''
]
]
]
])
])
pipeline {
agent any
stages {
stage('Build') {
steps {
script {
echo "Host:::: ${params.Host}"
}
}
}
}
}
Update
When you are calling a private repo, you need to send a Basic Auth header with the access token. So use the following groovy script instead.
def accessToken = "ACCESS_TOKEN".bytes.encodeBase64().toString()
def get = new URL("https://raw.githubusercontent.com/xxxx/something/hosts").openConnection();
get.setRequestProperty("authorization", "Basic " + accessToken)
def content = get.getInputStream().getText()
I have this rewrite in next.config.js:
async rewrites() {
return [
{
source: 'https://:postOrProgramId*.tikex-dev.com/',
destination: 'https://tikex-dev.com/:postOrProgramId*',
},
]
},
But it does not work.
I would do rewrite like this amazing-campaign-in-nyc.tikex-dev.com -> tikex-dev.com/amazing-campaign-in-nyc
Unfortunatelly when I run locally, I got this error, though I know it does not support localhost now.
`source` does not start with / for route {"source":"https://:postOrProgramId*.tikex-dev.com/","destination":"https://tikex-dev.com/:postOrProgramId*"}
Yes, source does not starts with / as I would expect first the protocol and then subdomain name comes. Is protocol + subdomain name not part of the source?
I am not sure, I need : before the variable and * after it? How can I express subdomain to subpath redirect?
I have an other reverseproxy rewrite. Do I have to worry about the two will interfere?
{
source: '/api/:path*',
destination: `${
true
? `https://api.tikex-${
process.env.NEXT_PUBLIC_ENVIRONMENT ?? 'dev'
}.com/`
: 'http://localhost:8080/'
}:path*`,
},
We use symfony with the platform api in docker and we have a problem with varnish.
For local development varnish works with the default.vcl configuration file (https://github.com/api-platform/api-platform/blob/master/api/docker/varnish/conf/default.vcl). When uploading to the server, varnish provides the error "Error (null) Backend fetch failed"
When disabling varnish and redirecting to nginx with php-fpm, api-platform works properly.
I increased the http_resp_hdr_len parameters to 131072 bytes (128K) and http_rasp_size to 10485760 bytes (10Mb), but it doesn't help, and the error remains.
docker command to start varnish:
CMD ["varnishd", "-F", "-f", "/etc/varnish/default.vcl", "-p", "http_resp_hdr_len=131072", "-p", "http_resp_size=10485760"]
Also the parameter - .first_byte_timeout = 600s; was added to default.vcl varnish
default.vcl
vcl 4.0;
import std;
backend default {
.host = "api";
.port = "80";
.first_byte_timeout = 600s;
# Health check
.probe = {
.url = "/";
.timeout = 5s;
.interval = 50s;
.window = 5;
.threshold = 3;
}
}
# Hosts allowed to send BAN requests
acl invalidators {
"localhost";
"php";
# local Kubernetes network
"10.0.0.0"/8;
"172.16.0.0"/12;
"192.168.0.0"/16;
}
sub vcl_recv {
if (req.restarts > 0) {
set req.hash_always_miss = true;
}
# Remove the "Forwarded" HTTP header if exists (security)
unset req.http.forwarded;
# To allow API Platform to ban by cache tags
if (req.method == "BAN") {
if (client.ip !~ invalidators) {
return (synth(405, "Not allowed"));
}
if (req.http.ApiPlatform-Ban-Regex) {
ban("obj.http.Cache-Tags ~ " + req.http.ApiPlatform-Ban-Regex);
return (synth(200, "Ban added"));
}
return (synth(400, "ApiPlatform-Ban-Regex HTTP header must be set."));
}
# For health checks
if (req.method == "GET" && req.url == "/healthz") {
return (synth(200, "OK"));
}
}
sub vcl_hit {
if (obj.ttl >= 0s) {
# A pure unadulterated hit, deliver it
return (deliver);
}
if (std.healthy(req.backend_hint)) {
# The backend is healthy
# Fetch the object from the backend
return (restart);
}
# No fresh object and the backend is not healthy
if (obj.ttl + obj.grace > 0s) {
# Deliver graced object
# Automatically triggers a background fetch
return (deliver);
}
# No valid object to deliver
# No healthy backend to handle request
# Return error
return (synth(503, "API is down"));
}
sub vcl_deliver {
# Don't send cache tags related headers to the client
unset resp.http.url;
# Comment the following line to send the "Cache-Tags" header to the client (e.g. to use CloudFlare cache tags)
unset resp.http.Cache-Tags;
}
sub vcl_backend_response {
# Ban lurker friendly header
set beresp.http.url = bereq.url;
# Add a grace in case the backend is down
set beresp.grace = 1h;
}
Please advise what might be a problem with varnish and how to make it work correctly?
The problem was solved as follows - I took the parameters for running varnish from thomasmoreaumaster (https://github.com/api-platform/api-platform/issues/1367).
CMD ["varnishd", "-F", "-f", "/etc/varnish/default.vcl", "-p", "http_resp_hdr_len=128k", "-p", "http_resp_size=128k", "-p", "http_req_hdr_len=64k", "-p", "workspace_backend=256k", "-p", "workspace_client=256k", "-p", "http_max_hdr=256"]
Also in the proxying nginx api-platform removed directory binding with ssl and formed an nginx-proxy with ssl enabled via volume.
Now varnish works well.
Thanks for your help and support.
I was trying to create some basic inspec tests to validate a set of HTTP URLs. The way I started is like this -
control 'http-url-checks' do
impact 1.0
title 'http-url-checks'
desc '
Specify the URLs which need to be up and working.
'
tag 'http-url-checks'
describe http('http://example.com') do
its('status') { should eq 200 }
its('body') { should match /abc/ }
its('headers.name') { should eq 'header' }
end
describe http('http://example.net') do
its('status') { should eq 200 }
its('body') { should match /abc/ }
its('headers.name') { should eq 'header' }
end
end
We notice that the URLs are hard-coded in the controls and isn't a lot of fun. I'd like to move them to some 'attributes' file of some sort and loop through them in the control file.
My attempt was to use the 'files' folder structure inside the profile.I created a file - httpurls.yml and had the following content in it -
- url: http://example.com
- url: http://example.net
..and in my control file, I had the construct -
my_urls = yaml(content: inspec.profile.file('httpurls.yml')).params
my_urls.each do |s|
describe http(s['url']) do
its('status') { should eq 200 }
end
end
However, when I execute the compliance profile, I get an error - 'httpurls.yml not found' (not sure about the exact error message though though). The following is the folder structure I had for my compliance profile.
What I am doing wrong?
Is there a better way to achieve what I am trying to do?
The secret is to use profile attributes, as defined near the bottom of this page:
https://www.inspec.io/docs/reference/profiles/
First, create a profile attributes YML file. I name mine profile-attribute.yml.
Second, put your array of values in the YML file, like so:
urls:
- http://example.com
- http://example.net
Third, create an attribute at the top of your InSpec tests:
my_urls = attribute('urls', description: 'The URLs that I am validating.')
Fourth, use your attribute in your InSpec test:
my_urls.each do |s|
describe http(s['url']) do
its('status') { should eq 200 }
end
end
Finally, when you call your InSpec test, point to your YML file using --attrs:
inspec exec mytest.rb --reporter=cli --attrs profile-attribute.yml
There is another way to do this using files (instead of the profile attributes and the --attrs flag). You can use JSON or YAML.
First, create the JSON and/or YAML file and put them in the files directory. A simple example of the JSON file might look like this:
{
"urls": ["https://www.google.com", "https://www.apple.com"]
}
And a simple example of the YAML file might look like this:
urls:
- https://www.google.com
- https://www.apple.com
Second, include code at the top of your InSpec file to read and parse the JSON and/or YAML, like so:
jsoncontent = inspec.profile.file("tmp.json")
jsonparams = JSON.parse(jsoncontent)
jsonurls = jsonparams['urls']
yamlcontent = inspec.profile.file("tmp.yaml")
yamlparams = YAML.load(yamlcontent)
yamlurls = yamlparams['urls']
Third, use the variables in your InSpec tests, like so:
jsonurls.each do |jsonurl|
describe http(jsonurl) do
puts "json url is " + jsonurl
its('status') { should eq 200 }
end
end
yamlurls.each do |yamlurl|
describe http(yamlurl) do
puts "yaml url is " + yamlurl
its('status') { should eq 200 }
end
end
(NOTE: the puts line is for debugging.)
The result is what you would expect:
json url is https://www.google.com
json url is https://www.apple.com
yaml url is https://www.google.com
yaml url is https://www.apple.com
Profile: InSpec Profile (inspec-file-test)
Version: 0.1.0
Target: local://
http GET on https://www.google.com
✔ status should eq 200
http GET on https://www.apple.com
✔ status should eq 200
http GET on https://www.google.com
✔ status should eq 200
http GET on https://www.apple.com
✔ status should eq 200
I need to validate agent like memory, processor, and port connectivity before running my other manifest files. So I've created a manifest like following by keeping global facts with and statement and exec resource.
class vc {
#Validateing Infra Before applying chnages
if $::facts['memorysize'] >= '4.00 GiB'and $::facts['processorcount'] >= 2 and Exec['port_connectivity'] {
notify { "Infra validated" : }
include vc::configs
}
else {
notify { "Infra not meeting requirements" : }
}
# Checking port connecitivity to puppet master
exec { 'port_connectivity':
command => 'New-Item c:\debug.txt -type file -Force',
unless => 'if((New-Object System.Net.Sockets.TcpClient ("linux-NAS-storage.com",6163)).connected -eq $true) { exit 1 }',
provider => powershell,
}
}
my theme is puppet should only execute if this if $::facts['memorysize'] >= '4.00 GiB'and $::facts['processorcount'] >= 2 and Exec['port_connectivity'] condition was successful. If exec command was successful and facter returns true then only it should execute, but following manifest executing individually without checking whether that exec statement true or not. My main purpose I need to validate ports before running puppet manifest. Can any please help