spring cloud gateway, Is Request size limit filter is available - spring-cloud

I am working in a project with spring-cloud-gateway. I see that the Request Size limitation filter is yet not available. But I need to develop it. Any idea , is it coming ? or should I start my own development.
I Know that it is difficult to get any answer, as except the developers there are a few persons who are working on it.

I have created a filter named RequestSizeGatewayFilterFactory, It is working fine for our application as of now. But not sure this can be a part of the spring-cloud-gateway project.
package com.api.gateway.somename.filter;
import org.springframework.cloud.gateway.filter.GatewayFilter;
import org.springframework.cloud.gateway.filter.factory.AbstractGatewayFilterFactory;
import org.springframework.http.HttpStatus;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.util.Assert;
import org.springframework.util.StringUtils;
/**
* This filter blocks the request, if the request size is more than
* the permissible size.The default request size is 5 MB
*
* #author Arpan
*/
public class RequestSizeGatewayFilterFactory
extends AbstractGatewayFilterFactory<RequestSizeGatewayFilterFactory.RequestSizeConfig> {
private static String PREFIX = "kMGTPE";
private static String ERROR = "Request size is larger than permissible limit." +
"Request size is %s where permissible limit is %s";
public RequestSizeGatewayFilterFactory() {
super(RequestSizeGatewayFilterFactory.RequestSizeConfig.class);
}
#Override
public GatewayFilter apply(RequestSizeGatewayFilterFactory.RequestSizeConfig requestSizeConfig) {
requestSizeConfig.validate();
return (exchange, chain) -> {
ServerHttpRequest request = exchange.getRequest();
String contentLength = request.getHeaders().getFirst("content-length");
if (!StringUtils.isEmpty(contentLength)) {
Long currentRequestSize = Long.valueOf(contentLength);
if (currentRequestSize > requestSizeConfig.getMaxSize()) {
exchange.getResponse().setStatusCode(HttpStatus.PAYLOAD_TOO_LARGE);
exchange.getResponse().getHeaders().add("errorMessage",
getErrorMessage(currentRequestSize, requestSizeConfig.getMaxSize()));
return exchange.getResponse().setComplete();
}
}
return chain.filter(exchange);
};
}
public static class RequestSizeConfig {
// 5 MB is the default request size
private Long maxSize = 5000000L;
public RequestSizeGatewayFilterFactory.RequestSizeConfig setMaxSize(Long maxSize) {
this.maxSize = maxSize;
return this;
}
public Long getMaxSize() {
return maxSize;
}
public void validate() {
Assert.isTrue(this.maxSize != null && this.maxSize > 0,
"maxSize must be greater than 0");
Assert.isInstanceOf(Long.class, maxSize, "maxSize must be a number");
}
}
private static String getErrorMessage(Long currentRequestSize, Long maxSize) {
return String.format(ERROR,
getHumanReadableByteCount(currentRequestSize),
getHumanReadableByteCount(maxSize));
}
private static String getHumanReadableByteCount(long bytes) {
int unit = 1000;
if (bytes < unit) return bytes + " B";
int exp = (int) (Math.log(bytes) / Math.log(unit));
String pre = Character.toString(PREFIX.charAt(exp - 1));
return String.format("%.1f %sB", bytes / Math.pow(unit, exp), pre);
}
}
And the configuration for the filter is:
When it works as a default filter:
spring:
application:
name: somename
cloud:
gateway:
default-filters:
- Hystrix=default
- RequestSize=7000000
When needs to be applied in some API
# ===========================================
- id: request_size_route
uri: ${test.uri}/upload
predicates:
- Path=/upload
filters:
- name: RequestSize
args:
maxSize: 5000000
Also you need to configure the bean in some component scanable class in your project, which is GatewayAutoConfiguration for the spring-cloud-gateway-core project.
#Bean
public RequestSizeGatewayFilterFactory requestSizeGatewayFilterFactory() {
return new RequestSizeGatewayFilterFactory();
}

Related

Getting the last message on a Gmail message thread in Katalon Studio

I'm using this plugin for Katalon Studio to access the last unread message from my testing Gmail account.
My email util class is like:
public final class SMDEmailUtils {
public static final String MainInboxFolder = "INBOX";
public static final String SpamFolder = "[Gmail]/Spam";
public static String GetMainEmail() {
if (!GeneralWebUIUtils.GlobalVariableExists('emailID'))
return "dev#example.com";
return GlobalVariable.emailID.toString();
}
public static String ExtractSignUpLink() {
final String folderName = this.GetNewMessageFolderName(30, FailureHandling.STOP_ON_FAILURE);
return this.ProcessHTML(this.GetNewMessage(folderName), "//a[.//div[#class = 'sign-mail-btn-text']]/#href");
}
public static String GetNewMessageFolderName(int timeOut,
FailureHandling failureHandling = FailureHandling.STOP_ON_FAILURE) {
final long startTime = System.currentTimeMillis()
final Map<String, Integer> folderMessageCountDict = [
(this.MainInboxFolder) : this.GetMessageCount(this.MainInboxFolder),
(this.SpamFolder) : this.GetMessageCount(this.SpamFolder),
];
while (System.currentTimeMillis() < startTime + 1000 * timeOut) {
final String folderName = folderMessageCountDict.findResult({String folderName, int initialMessageCount ->
if (initialMessageCount < this.GetMessageCount(folderName))
return folderName;
return null;
})
if (folderName != null)
return folderName;
// TODO: we shouldn't have to do some hard-coded suspension of the runtime. We need to close the store somehow
Thread.sleep(1000 * 2);
}
throw new StepFailedException("Failed to find a folder with a new message in it after ${(System.currentTimeMillis() - startTime) / 1000} seconds");
}
public static int GetMessageCount(String folderName) {
return Gmail.getEmailsCount(this.GetMainEmail(), GlobalVariable.emailPassword, folderName);
}
public static String GetNewMessage(String folderName) {
return Gmail.readLatestEMailBodyContent(this.GetMainEmail(), GlobalVariable.emailPassword, folderName);
}
/**
* **NOTE**: forked from https://stackoverflow.com/a/2269464/2027839 , and then refactored
*
* Processes HTML, using XPath
*
* #param html
* #param xpath
* #return the result
*/
public static String ProcessHTML(String html, String xpath) {
final String properHTML = this.ToProperHTML(html);
final Element document = DocumentBuilderFactory.newInstance()
.newDocumentBuilder()
.parse(new ByteArrayInputStream( properHTML.bytes ))
.documentElement;
return XPathFactory.newInstance()
.newXPath()
.evaluate( xpath, document );
}
private static String ToProperHTML(String html) {
// SOURCE: https://stackoverflow.com/a/19125599/2027839
String properHTML = html.replaceAll( "(&(?!amp;))", "&" );
if (properHTML.contains('<!DOCTYPE html'))
return properHTML;
return """<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head></head>
<body>
${properHTML}
</body>
</html>
""";
}
}
My use case of that is the following:
a test member lead, whose email forwards to my testing email ([myTestingEmailName]+[memberLeadName]#gmail.com), gets a link to an agreement to sign
on successful signature, the physician, whose email also forwards to my testing email ([myTestingEmailName]+[physicianName]#gmail.com), gets a link to an agreement to sign
Step 1 works, the link gets extracted successfully via SMDEmailUtils.ExtractSignUpLink() .
However, when it is the physician's turn to sign, that same line of code doesn't work. It's giving me the link from the first email message (the one meant for the recipient in step 1, that was already signed).
I check out my inbox manually, and see this:
The AUT sent both email messages on the same thread, but the plugin can only handle the first message on the thread!
How do I handle this?

Unexpected java.util.ConcurrentModificationException in GWT emulated AbstractHashMap

We are experiencing a java.util.ConcurrentModificationException using gwt 2.8.0, while calling iter.next() in the GWT emulated AbstractHashMap class (See stack trace and our CallbackTimer class below). The lowest point in the trace involving our code is on line 118, in the method private void tick(), a call to iter.next().
Drilling into trace, I see AbstractHashMap:
#Override
public Entry<K, V> next() {
checkStructuralChange(AbstractHashMap.this, this);
checkElement(hasNext());
last = current;
Entry<K, V> rv = current.next();
hasNext = computeHasNext();
return rv;
}
calling ConcurrentModificationDetector.checkStructuralChange:
public static void checkStructuralChange(Object host, Iterator<?> iterator) {
if (!API_CHECK) {
return;
}
if (JsUtils.getIntProperty(iterator, MOD_COUNT_PROPERTY)
!= JsUtils.getIntProperty(host, MOD_COUNT_PROPERTY)) {
throw new ConcurrentModificationException();
}
}
My understanding of the purpose of ConcurrentModificationException is to avoid having the collection change while it is being iterated over. I don't think iter.next() would fall into that category. Further, the only places I see the collection changing during iteration do so through the iterator itself. Am I missing something here? Any help would be appreciated!
Our Stack Trace:
java.util.ConcurrentModificationException
at Unknown.Throwable_1_g$(Throwable.java:61)
at Unknown.Exception_1_g$(Exception.java:25)
at Unknown.RuntimeException_1_g$(RuntimeException.java:25)
at Unknown.ConcurrentModificationException_1_g$(ConcurrentModificationException.java:25)
at Unknown.checkStructuralChange_0_g$(ConcurrentModificationDetector.java:54)
at Unknown.next_79_g$(AbstractHashMap.java:106)
at Unknown.next_78_g$(AbstractHashMap.java:105)
at Unknown.next_81_g$(AbstractMap.java:217)
at Unknown.tick_0_g$(CallbackTimer.java:118)
at Unknown.run_47_g$(CallbackTimer.java:41)
at Unknown.fire_0_g$(Timer.java:135)
at Unknown.anonymous(Timer.java:139)
at Unknown.apply_65_g$(Impl.java:239)
at Unknown.entry0_0_g$(Impl.java:291)
at Unknown.anonymous(Impl.java:77)
The source code for CallbackTimer.java is here:
package com.XXXXX.common.gwt.timer;
import java.util.ConcurrentModificationException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.logging.Level;
import java.util.logging.Logger;
import com.google.common.base.Optional;
import com.google.gwt.user.client.Timer;
/**
* A {#link Timer} wrapper which allows for the registration of callbacks to be invoked after a given number of ticks.
* The timer will only run if at least one {#link TickCallback} is currently registered and will stop running when all
* callbacks have been unregistered.
*
* The intent of this class is to reduce overhead by allowing all callbacks in a GWT application to use the same
* Javascript timer.
*/
public class CallbackTimer
{
private static final Logger LOGGER = Logger.getLogger(CallbackTimer.class.getName());
private static final int MILLIS_IN_SEC = 1000;
private Timer timer;
private Map<Object, TickCallback> callbackRegistry = new HashMap<>();
public CallbackTimer()
{
timer = new Timer()
{
#Override
public void run()
{
try
{
tick();
}
catch(ConcurrentModificationException concurrentModificationException)
{
LOGGER.log(Level.WARNING, "Concurrent Modification Exception in " +
"CallbackTimer.tick()", concurrentModificationException);
}
}
};
}
public void registerCallback(Object key, TickCallback callback)
{
if (callbackRegistry.containsKey(key))
{
LOGGER.fine("Key " + key.toString() + " is being overwritten with a new callback.");
}
callbackRegistry.put(key, callback);
callback.markStartTime();
LOGGER.finer("Key " + key.toString() + " registered.");
if (!timer.isRunning())
{
startTimer();
}
}
public void unregisterCallback(Object key)
{
if (callbackRegistry.containsKey(key))
{
callbackRegistry.remove(key);
LOGGER.finer("Key " + key.toString() + " unregistered.");
if (callbackRegistry.isEmpty())
{
stopTimer();
}
}
else
{
LOGGER.info("Attempted to unregister key " + key.toString() + ", but this key has not been registered.");
}
}
private void unregisterCallback(Iterator<Object> iter, Object key)
{
iter.remove();
LOGGER.finer("Key " + key.toString() + " unregistered.");
if (callbackRegistry.isEmpty())
{
stopTimer();
}
}
public boolean keyIsRegistered(Object key)
{
return callbackRegistry.containsKey(key);
}
public TickCallback getCallback(Object key)
{
if (keyIsRegistered(key))
{
return callbackRegistry.get(key);
}
else
{
LOGGER.fine("Key " + key.toString() + " is not registered; returning null.");
return null;
}
}
private void tick()
{
long fireTimeMillis = System.currentTimeMillis();
Iterator<Object> iter = callbackRegistry.keySet().iterator();
while (iter.hasNext())
{
Object key = iter.next();//Lowest point in stack for our code
TickCallback callback = callbackRegistry.get(key);
if (callback.isFireTime(fireTimeMillis))
{
if (Level.FINEST.equals(LOGGER.getLevel()))
{
LOGGER.finest("Firing callback for key " + key.toString());
}
callback.onTick();
callback.markLastFireTime();
}
if (callback.shouldTerminate())
{
LOGGER.finer("Callback for key " + key.toString() +
" has reached its specified run-for-seconds and will now be unregistered.");
unregisterCallback(iter, key);
}
}
}
private void startTimer()
{
timer.scheduleRepeating(MILLIS_IN_SEC);
LOGGER.finer(this + " started.");
}
private void stopTimer()
{
timer.cancel();
LOGGER.finer(this + " stopped.");
}
/**
* A task to run on a given interval, with the option to specify a maximum number of seconds to run.
*/
public static abstract class TickCallback
{
private long intervalMillis;
private long startedAtMillis;
private long millisRunningAtLastFire;
private Optional<Long> runForMillis;
/**
* #param intervalSeconds
* The number of seconds which must elapse between each invocation of {#link #onTick()}.
* #param runForSeconds
* An optional maximum number of seconds to run for, after which the TickCallback will be eligible
* to be automatically unregistered. Pass {#link Optional#absent()} to specify that the TickCallback
* must be manually unregistered. Make this value the same as {#param intervalSeconds} to run the
* callback only once.
*/
public TickCallback(int intervalSeconds, Optional<Integer> runForSeconds)
{
this.intervalMillis = intervalSeconds * MILLIS_IN_SEC;
this.runForMillis = runForSeconds.isPresent() ?
Optional.of((long)runForSeconds.get() * MILLIS_IN_SEC) : Optional.<Long>absent();
}
private void markStartTime()
{
millisRunningAtLastFire = 0;
startedAtMillis = System.currentTimeMillis();
}
private void markLastFireTime()
{
millisRunningAtLastFire += intervalMillis;
}
private boolean isFireTime(long nowMillis)
{
return nowMillis - (startedAtMillis + millisRunningAtLastFire) >= intervalMillis;
}
private boolean shouldTerminate()
{
return runForMillis.isPresent() && System.currentTimeMillis() - startedAtMillis >= runForMillis.get();
}
/**
* A callback to be run every time intervalSeconds seconds have past since this callback was registered.
*/
public abstract void onTick();
}
}
Update 2017-06-08
I ended up following walen's first suggestion. I did not see where SimpleEventBus was the right tool for this specific job. I did, however, shamelessly steal SEBs method of integrating newly added/removed callbacks:
package com.XXXXX.common.gwt.timer;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.logging.Level;
import java.util.logging.Logger;
import com.google.common.base.Optional;
import com.google.gwt.user.client.Command;
import com.google.gwt.user.client.Timer;
/**
* A {#link Timer} wrapper which allows for the registration of callbacks to be invoked after a given number of ticks.
* The timer will only run if at least one {#link TickCallback} is currently registered and will stop running when all
* callbacks have been unregistered.
*
* The intent of this class is to reduce overhead by allowing all callbacks in a GWT application to use the same
* Javascript timer.
*/
public class CallbackTimer
{
private static final Logger LOGGER = Logger.getLogger(CallbackTimer.class.getName());
private static final int MILLIS_IN_SEC = 1000;
private Timer timer;
private Map<Object, TickCallback> callbackRegistry = new HashMap<>();
private List<Command> deferredDeltas = new ArrayList<>();
public CallbackTimer()
{
timer = new Timer()
{
#Override
public void run()
{
tick();
}
};
}
public void registerCallback(final Object key, final TickCallback callback)
{
deferredDeltas.add(new Command()
{
#Override
public void execute()
{
activateCallback(key, callback);
}
});
if (!timer.isRunning())
{
startTimer();
}
}
private void activateCallback(Object key, TickCallback callback)
{
if (callbackRegistry.containsKey(key))
{
LOGGER.fine("Key " + key.toString() + " is being overwritten with a new callback.");
}
callbackRegistry.put(key, callback);
callback.markStartTime();
LOGGER.finer("Key " + key.toString() + " registered.");
}
public void unregisterCallback(final Object key)
{
deferredDeltas.add(new Command()
{
#Override
public void execute()
{
deactivateCallback(key);
}
});
}
private void deactivateCallback(Object key)
{
if (callbackRegistry.containsKey(key))
{
callbackRegistry.remove(key);
LOGGER.fine("Key " + key.toString() + " unregistered.");
if (callbackRegistry.isEmpty())
{
stopTimer();
}
}
else
{
LOGGER.info("Attempted to unregister key " + key.toString() + ", but this key has not been registered.");
}
}
private void handleQueuedAddsAndRemoves()
{
for (Command c : deferredDeltas)
{
c.execute();
}
deferredDeltas.clear();
}
public boolean keyIsRegistered(Object key)
{
return callbackRegistry.containsKey(key);
}
private void tick()
{
handleQueuedAddsAndRemoves();
long fireTimeMillis = System.currentTimeMillis();
for (Map.Entry<Object, TickCallback> objectTickCallbackEntry : callbackRegistry.entrySet())
{
Object key = objectTickCallbackEntry.getKey();
TickCallback callback = objectTickCallbackEntry.getValue();
if (callback.isFireTime(fireTimeMillis))
{
if (Level.FINEST.equals(LOGGER.getLevel()))
{
LOGGER.finest("Firing callback for key " + key.toString());
}
callback.onTick();
callback.markLastFireTime();
}
if (callback.shouldTerminate())
{
LOGGER.finer("Callback for key " + key.toString() +
" has reached its specified run-for-seconds and will now be unregistered.");
unregisterCallback(key);
}
}
}
private void startTimer()
{
timer.scheduleRepeating(MILLIS_IN_SEC);
LOGGER.finer(this + " started.");
}
private void stopTimer()
{
timer.cancel();
LOGGER.finer(this + " stopped.");
}
/**
* A task to run on a given interval, with the option to specify a maximum number of seconds to run.
*/
public static abstract class TickCallback
{
private long intervalMillis;
private long startedAtMillis;
private long millisRunningAtLastFire;
private Optional<Long> runForMillis;
/**
* #param intervalSeconds The number of seconds which must elapse between each invocation of {#link #onTick()}.
* #param runForSeconds An optional maximum number of seconds to run for, after which the TickCallback will be
* eligible
* to be automatically unregistered. Pass {#link Optional#absent()} to specify that the TickCallback
* must be manually unregistered. Make this value the same as {#param intervalSeconds} to run the
* callback only once.
*/
protected TickCallback(int intervalSeconds, Optional<Integer> runForSeconds)
{
this.intervalMillis = intervalSeconds * MILLIS_IN_SEC;
this.runForMillis = runForSeconds.isPresent() ?
Optional.of((long) runForSeconds.get() * MILLIS_IN_SEC) : Optional.<Long>absent();
}
private void markStartTime()
{
millisRunningAtLastFire = 0;
startedAtMillis = System.currentTimeMillis();
}
private void markLastFireTime()
{
millisRunningAtLastFire += intervalMillis;
}
private boolean isFireTime(long nowMillis)
{
return nowMillis - (startedAtMillis + millisRunningAtLastFire) >= intervalMillis;
}
private boolean shouldTerminate()
{
return runForMillis.isPresent() && System.currentTimeMillis() - startedAtMillis >= runForMillis.get();
}
/**
* A callback to be run every time intervalSeconds seconds have past since this callback was registered.
*/
public abstract void onTick();
}
}
Your problem seems to be that new items (new keys) are being added to the map at the same time that the tick() method is trying to traverse its keySet.
Modifying a collection in any way while traversing it will throw a ConcurrentModificationException.
Using an iterator only lets you avoid that when removing items, but since there's no iterator.add() method, you can't add items safely.
If this was server-side code, you could use a ConcurrentHashMap, which guarantees that its iterator won't throw an exception in such case (at the expense of not guaranteeing that every item will be traversed, if it was added after iterator creation).
But ConcurrentHashMap is not (yet) supported by GWT's JRE emulation library, so you can't use it in client-side code.
You'll need to come up with a different way of adding items to your CallbackRegistry.
You could, for example, change your registerCallback() method so new items are added to a list / queue instead of the map, and then have the tick() method move those items from the queue to the map after it's done traversing the existing ones.
Alternatively, you could use SimpleEventBus as pointed out in Thomas Broyer's comment.

Kryonet RMI performance

I was trying to run a performance test for Kryonet RMI, the test results are not convincing. However I think I may not be doing things in the right way. Can I get some feedback on the code below.
SERVER
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryonet.Connection;
import com.esotericsoftware.kryonet.Listener;
import com.esotericsoftware.kryonet.Server;
import com.esotericsoftware.kryonet.rmi.ObjectSpace;
public class Razor {
static int port = 2551;
static String send = null;
// Data Creator, creates data of size s.getBytes().length * 10
static String createMsg(String s){
StringBuilder sb = new StringBuilder();
for (int i = 0 ; i < 10; i++){
sb.append(s);
}
System.out.println(sb.toString().getBytes().length);
return sb.toString();
}
// Test Interface
public static interface TestInterface {
public String getName (String name);
}
//Class implementing the test interface.
static private class TestImpl implements TestInterface {
public String getName (String name) {
return send;
}
}
public static void main (String[] args) throws Exception {
// Creating data of size 160 bytes
send = createMsg("FooAndBarAndLazy");
Server server = new Server();
Kryo kryo = server.getKryo();
ObjectSpace.registerClasses(kryo);
kryo.register(TestInterface.class);
server.start();
server.bind(port);
System.out.println("Server started on " + port);
TestImpl test = new TestImpl();
final ObjectSpace objectSpace = new ObjectSpace();
objectSpace.register(123, test);
server.addListener(new Listener() {
public void connected (final Connection connection) {
objectSpace.addConnection(connection);
}
});
}
}
CLIENT
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryonet.Client;
import com.esotericsoftware.kryonet.rmi.ObjectSpace;
public class TBone {
static int port = 2551;
// Bytes to MB converter
public static float b2mb(long bytes){
float bb = bytes;
float ll = 1024 * 1024;
return bb/ll;
}
// Nono Seconds to seconds converter
public static float ns2s(long nsecs){
float f = nsecs;
float t = 1000000000;
return f/t;
}
public static void main (String[] args) throws Exception {
Client client = new Client();
Kryo kryo = client.getKryo();
ObjectSpace.registerClasses(kryo);
kryo.register(Razor.TestInterface.class);
client.start();
client.connect(50000, "localhost", port);
Razor.TestInterface test = ObjectSpace.getRemoteObject(client, 123, Razor.TestInterface.class);
String profile = null;
int bytes = 0;
long stime = System.nanoTime();
for (int i = 0; i < 1000; i++){
profile = test.getName("");
bytes = bytes + profile.getBytes().length;
if (i %100 == 0) System.out.println("Done : " + i);
}
long etime = System.nanoTime();
System.out.println("Total bytes(MB) : " + b2mb(bytes)+" , " + "Total time : " + ns2s((etime-stime))+" seconds");
client.stop();
}
}
RESULT
Done : 0
Done : 100
Done : 200
Done : 300
Done : 400
Done : 500
Done : 600
Done : 700
Done : 800
Done : 900
Total bytes(MB) : 0.15258789 , Total time : 26.139627 seconds
I would imagine way more performance. Is this a valid test?
This isn't a valid test of performance because even in a localhost request you have a strong latency and your test program wait for the server response before sending next request (as a rmi method call is by nature blocking).
Since you do it 900 time, even without any java processing you have to wait :
number_of_request * travel_time(latency) * 2(one travel for the request and one for the response).
So in your case :
900 * 0.03seconds (ping localhost to see your local latency) ~= 27 seconds

jnetpcap get html web page source code

I'm using jnetpcap to analyzing packet. I want to get html web page source code.
However, when I use html.page() to get source code I get some messy code which look like binary code. Can anyone help me? How to solve it?
Not a jnetpcap expert, but I have been using this class for a while and it seems to work. It actually gets many HTTP fields, including its payload.
package br.com.mvalle.ids.sniffer;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import org.jnetpcap.Pcap;
import org.jnetpcap.PcapIf;
import org.jnetpcap.packet.PcapPacket;
import org.jnetpcap.packet.PcapPacketHandler;
import org.jnetpcap.packet.format.FormatUtils;
import org.jnetpcap.protocol.network.Ip4;
import org.jnetpcap.protocol.tcpip.Http;
import org.jnetpcap.protocol.tcpip.Tcp;
public class Sniffer {
private List<PcapIf> alldevs = new ArrayList<PcapIf>(); // Will be filled with NICs
private StringBuilder errbuf = new StringBuilder(); // For any error msgs
private PcapIf selectedDevice;
private Pcap pcap;
private PcapPacketHandler<String> jpacketHandler;
public Sniffer(){
listDevices();
selectedDevice = selectDevice(1);
openDevice(selectedDevice);
packetHandler();
capturePackets();
}
public void listDevices(){
int r = Pcap.findAllDevs(alldevs, errbuf);
if (r == Pcap.NOT_OK || alldevs.isEmpty()) {
System.err.printf("Can't read list of devices, error is %s", errbuf.toString());
return;
}
System.out.println("Network devices found:");
int i = 0;
for (PcapIf device : alldevs) {
String description =
(device.getDescription() != null) ? device.getDescription()
: "No description available";
System.out.printf("#%d: %s [%s]\n", i++, device.getName(), description);
}
}
private PcapIf selectDevice(int deviceId){
PcapIf device = alldevs.get(1); // We know we have atleast 1 device (parameter changed from 0 to 1)
System.out
.printf("\nChoosing '%s' on your behalf:\n",
(device.getDescription() != null) ? device.getDescription()
: device.getName());
return device;
}
private void openDevice (PcapIf device){
int snaplen = 64 * 1024; // Capture all packets, no trucation
int flags = Pcap.MODE_PROMISCUOUS; // capture all packets
int timeout = 10 * 1000; // 10 seconds in millis
pcap =
Pcap.openLive(device.getName(), snaplen, flags, timeout, errbuf);
if (pcap == null) {
System.err.printf("Error while opening device for capture: "
+ errbuf.toString());
return;
}
}
private void packetHandler(){
jpacketHandler = new PcapPacketHandler<String>() {
Http httpheader = new Http();
public void nextPacket(PcapPacket packet, String user) {
if(packet.hasHeader(httpheader)){
System.out.println(httpheader.toString());
if(httpheader.hasPayload()){
System.out.println("HTTP payload: (string length is "
+new String(httpheader.getPayload()).length()+")");
System.out.println(new String(httpheader.getPayload()));
System.out.println("HTTP truncated? "
+httpheader.isPayloadTruncated());
}
//System.out.println(packet.toString());
}}
};
}
private void capturePackets(){
pcap.loop(pcap.LOOP_INFINITE , jpacketHandler, "Received Packet");
pcap.close();
}
}
Hope it helps.

How to change GWT Place URL from the default ":" to "/"?

By default, a GWT Place URL consists of the Place's simple class name (like "HelloPlace") followed by a colon (:) and the token returned by the PlaceTokenizer.
My question is how can I change ":" to be "/"?
I just made my own PlaceHistoryMapper that directly implements the interface instead of using AbstractPlaceHistoryMapper:
public class AppPlaceHistoryMapper implements PlaceHistoryMapper
{
String delimiter = "/";
#Override
public Place getPlace(String token)
{
String[] tokens = token.split(delimiter, 2);
if (tokens[0].equals("HelloPlace"))
...
}
#Override
public String getToken(Place place)
{
if (place instanceof HelloPlace)
{
return "HelloPlace" + delimiter + whatever;
}
else ...
}
}
It's certainly extra code to write, but you can control your url structure all in one place, and use slashes instead of colons!
Here is how to customize the delimiter, while using the standard GWT places. (PlaceHistoryMapper)
Nothing else needs to be changed; it works with the standard way of using Places and Tokenizers.
Insert into gwt.xml
<generate-with
class="com.google.gwt.place.rebind.CustomPlaceHistoryMapperGenerator">
<when-type-assignable class="com.google.gwt.place.shared.PlaceHistoryMapper" />
</generate-with>
Add CustomAbstractPlaceHistoryMapper
package com.siderakis.client.mvp;
import com.google.gwt.place.impl.AbstractPlaceHistoryMapper;
import com.google.gwt.place.shared.Place;
import com.google.gwt.place.shared.PlaceTokenizer;
public abstract class CustomAbstractPlaceHistoryMapper extends AbstractPlaceHistoryMapper {
public final static String DELIMITER = "/";
public static class CustomPrefixAndToken extends PrefixAndToken {
public CustomPrefixAndToken(String prefix, String token) {
super(prefix, token);
assert prefix != null && !prefix.contains(DELIMITER);
}
#Override
public String toString() {
return (prefix.length() == 0) ? token : prefix + DELIMITER + token;
}
}
#Override
public Place getPlace(String token) {
int colonAt = token.indexOf(DELIMITER);
String initial;
String rest;
if (colonAt >= 0) {
initial = token.substring(0, colonAt);
rest = token.substring(colonAt + 1);
} else {
initial = "";
rest = token;
}
PlaceTokenizer tokenizer = getTokenizer(initial);
if (tokenizer != null) {
return tokenizer.getPlace(rest);
}
return null;
}
}
Add CustomPlaceHistoryMapperGenerator
/*
* Copyright 2010 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/
package com.google.gwt.place.rebind;
import java.io.PrintWriter;
import com.siderakis.client.mvp.CustomAbstractPlaceHistoryMapper;
import com.siderakis.client.mvp.CustomAbstractPlaceHistoryMapper.CustomPrefixAndToken;
import com.google.gwt.core.client.GWT;
import com.google.gwt.core.ext.Generator;
import com.google.gwt.core.ext.GeneratorContext;
import com.google.gwt.core.ext.TreeLogger;
import com.google.gwt.core.ext.UnableToCompleteException;
import com.google.gwt.core.ext.typeinfo.JClassType;
import com.google.gwt.core.ext.typeinfo.JMethod;
import com.google.gwt.place.shared.Place;
import com.google.gwt.place.shared.PlaceTokenizer;
import com.google.gwt.user.rebind.ClassSourceFileComposerFactory;
import com.google.gwt.user.rebind.SourceWriter;
/**
* Generates implementations of
* {#link com.google.gwt.place.shared.PlaceHistoryMapper PlaceHistoryMapper}.
*/
public class CustomPlaceHistoryMapperGenerator extends Generator {
private PlaceHistoryGeneratorContext context;
#Override
public String generate(TreeLogger logger, GeneratorContext generatorContext,
String interfaceName) throws UnableToCompleteException {
context = PlaceHistoryGeneratorContext.create(logger,
generatorContext.getTypeOracle(), interfaceName);
if (context == null) {
return null;
}
PrintWriter out = generatorContext.tryCreate(logger, context.packageName,
context.implName);
if (out != null) {
generateOnce(generatorContext, context, out);
}
return context.packageName + "." + context.implName;
}
private void generateOnce(GeneratorContext generatorContext, PlaceHistoryGeneratorContext context,
PrintWriter out) throws UnableToCompleteException {
TreeLogger logger = context.logger.branch(TreeLogger.DEBUG, String.format(
"Generating implementation of %s", context.interfaceType.getName()));
ClassSourceFileComposerFactory f = new ClassSourceFileComposerFactory(
context.packageName, context.implName);
String superClassName = String.format("%s<%s>",
CustomAbstractPlaceHistoryMapper.class.getSimpleName(),
context.factoryType == null ? "Void" : context.factoryType.getName());
f.setSuperclass(superClassName);
f.addImplementedInterface(context.interfaceType.getName());
f.addImport(CustomAbstractPlaceHistoryMapper.class.getName());
f.addImport(context.interfaceType.getQualifiedSourceName());
f.addImport(CustomAbstractPlaceHistoryMapper.class.getCanonicalName());
if (context.factoryType != null) {
f.addImport(context.factoryType.getQualifiedSourceName());
}
f.addImport(Place.class.getCanonicalName());
f.addImport(PlaceTokenizer.class.getCanonicalName());
f.addImport(CustomPrefixAndToken.class.getCanonicalName());
f.addImport(GWT.class.getCanonicalName());
SourceWriter sw = f.createSourceWriter(generatorContext, out);
sw.println();
writeGetPrefixAndToken(context, sw);
sw.println();
writeGetTokenizer(context, sw);
sw.println();
sw.outdent();
sw.println("}");
generatorContext.commit(logger, out);
}
private void writeGetPrefixAndToken(PlaceHistoryGeneratorContext context,
SourceWriter sw) throws UnableToCompleteException {
sw.println("protected CustomPrefixAndToken getPrefixAndToken(Place newPlace) {");
sw.indent();
for (JClassType placeType : context.getPlaceTypes()) {
String placeTypeName = placeType.getQualifiedSourceName();
String prefix = context.getPrefix(placeType);
sw.println("if (newPlace instanceof " + placeTypeName + ") {");
sw.indent();
sw.println(placeTypeName + " place = (" + placeTypeName + ") newPlace;");
JMethod getter = context.getTokenizerGetter(prefix);
if (getter != null) {
sw.println(String.format("return new CustomPrefixAndToken(\"%s\", "
+ "factory.%s().getToken(place));", escape(prefix),
getter.getName()));
} else {
sw.println(String.format(
"PlaceTokenizer<%s> t = GWT.create(%s.class);", placeTypeName,
context.getTokenizerType(prefix).getQualifiedSourceName()));
sw.println(String.format("return new CustomPrefixAndToken(\"%s\", "
+ "t.getToken((%s) place));", escape(prefix), placeTypeName));
}
sw.outdent();
sw.println("}");
}
sw.println("return null;");
sw.outdent();
sw.println("}");
}
private void writeGetTokenizer(PlaceHistoryGeneratorContext context,
SourceWriter sw) throws UnableToCompleteException {
sw.println("protected PlaceTokenizer getTokenizer(String prefix) {");
sw.indent();
for (String prefix : context.getPrefixes()) {
JMethod getter = context.getTokenizerGetter(prefix);
sw.println("if (\"" + escape(prefix) + "\".equals(prefix)) {");
sw.indent();
if (getter != null) {
sw.println("return factory." + getter.getName() + "();");
} else {
sw.println(String.format("return GWT.create(%s.class);",
context.getTokenizerType(prefix).getQualifiedSourceName()));
}
sw.outdent();
sw.println("}");
}
sw.println("return null;");
sw.outdent();
sw.println("}");
sw.outdent();
}
}
Good question. The problem is, that this is hard-coded into AbstractPlaceHistoryMapper:
AbstractPlaceHistoryMapper.PrefixAndToken.toString():
return (prefix.length() == 0) ? token : prefix + ":" + token;
AbstractPlaceHistoryMapper.getPlace(String token):
int colonAt = token.indexOf(':');
...
And AbstractPlaceHistoryMapper is hard-coded into PlaceHistoryMapperGenerator.
It would probably be possible to exchange the generator by supplying your own module xml file, and reconfiguring the binding, but overall, I would consider this as "basically not configurable". (But see Riley's answer for a good alternative without declarative Tokenizer configuration!)
)
I really appreciated the answers, thanks SO for making me optimize my time (I was following in debugger where my getToken() method was called, and it's getting through listeners, handlers, magicians and funny stuff like that :-)
So thinking of the HistoryMapper is perfect, but the solution if you want to add a crawler is a lot simpler since you just need to add an extra '!' after the bookmark hash #!
So it is enough to simply decorate the original result with an extra character.
public class PlaceHistoryMapperDecorator implements PlaceHistoryMapper {
private static final String CRAWLER_PREFIX = "!";
protected PlaceHistoryMapper delegateHistoryMapper;
public PlaceHistoryMapperDecorator(PlaceHistoryMapper delegateHistoryMapper) {
this.delegateHistoryMapper = delegateHistoryMapper;
}
#Override
public Place getPlace(String token) {
String cleanToken = token;
if (token.startsWith(CRAWLER_PREFIX))
cleanToken = token.substring(CRAWLER_PREFIX.length());
else {
if (token.length() > 0)
System.err.println("there might be an error: can't find crawler prefix in " + token);
}
return delegateHistoryMapper.getPlace(cleanToken);
}
#Override
public String getToken(Place place) {
return CRAWLER_PREFIX + delegateHistoryMapper.getToken(place);
}
}
Then you pass that new instance to your PlaceHistoryHandler and that's it
PlaceHistoryMapperDecorator historyMapperDecorator = new PlaceHistoryMapperDecorator((PlaceHistoryMapper) GWT.create(AppPlaceHistoryMapper.class));
PlaceHistoryHandler historyHandler = new PlaceHistoryHandler(historyMapperDecorator);
I tested it before posting this message, it works fine :-)