Apache MINA SSHD session echoes characters twice when using ProcessShellFactory, control characters don't work - echo

I'm running OpenJDK 14 on macOS 10.15.7. I'm doing some proof-of-concept code establishing an SSH server with Apache Mina SSHD and then connecting to it. Here's what I have:
import java.io.IOException;
import java.nio.file.Paths;
import java.util.Arrays;
import org.apache.sshd.common.cipher.BuiltinCiphers;
import org.apache.sshd.common.util.logging.AbstractLoggingBean;
import org.apache.sshd.server.ServerBuilder;
import org.apache.sshd.server.SshServer;
import org.apache.sshd.server.auth.AsyncAuthException;
import org.apache.sshd.server.auth.password.PasswordAuthenticator;
import org.apache.sshd.server.auth.password.PasswordChangeRequiredException;
import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
import org.apache.sshd.server.session.ServerSession;
import org.apache.sshd.server.shell.InteractiveProcessShellFactory;
import org.apache.sshd.server.shell.ProcessShellFactory;
public class FunctionalTest
{
private static class TestAuthenticator
extends AbstractLoggingBean
implements PasswordAuthenticator
{
#Override
public boolean authenticate(String username, String password, ServerSession session)
throws PasswordChangeRequiredException, AsyncAuthException
{
if ("test".equals(username) && "foobar".equals(password))
{
this.log.info("authenticate({}[{}]: accepted", username, session);
return true;
}
this.log.warn("authenticate({}[{}]: rejected", username, session);
return false;
}
}
public static void main(String... args) throws IOException
{
SshServer sshd = SshServer.setUpDefaultServer();
sshd.setHost("0.0.0.0");
sshd.setPort(1022);
sshd.setShellFactory(InteractiveProcessShellFactory.INSTANCE);
sshd.setPasswordAuthenticator(new TestAuthenticator());
sshd.setCipherFactories(Arrays.asList(BuiltinCiphers.aes256ctr, BuiltinCiphers.aes192ctr));
sshd.setKeyExchangeFactories(ServerBuilder.setUpDefaultKeyExchanges(false));
sshd.setKeyPairProvider(new SimpleGeneratorHostKeyProvider(Paths.get("key.ser")));
sshd.start();
try
{
Thread.sleep(3_600_000);
}
catch(InterruptedException e)
{
System.out.println("Caught interrupt ... stopping server.");
sshd.stop(true);
}
}
}
When I start this, I can ssh -p 1022 test#localhost with the password foobar and it works. After successful authentication, I first see this:
sh: no job control in this shell
Then at the prompt, characters I type (including newlines) are echoed twice instead of once, resulting in everything being dduupplliiccaatteedd:
williamsn:mina-test williamsn$ llss --aall
total 24
... (list of files)
williamsn:mina-test williamsn$ eecchhoo hheelllloo
hello
williamsn:mina-test williamsn$
Additionally, if I run an interactive command like top, it doesn't recognize my inputs, and control characters don't work. ttoopp starts (though its output is ugly and additive instead of replacing the screen), but if I type q to exit (q is not echoed twice in this case), top does not exit in response to the q. It just keeps going. ctrl+c also does not work—top just keeps going. The only way to exit top is to kill my ssh process or shut down the MINA server.
I feel like I must be doing something terribly wrong here. Thoughts?

The "no job control" message indicates that the spawned shell Is not in a full interactive mode, and the double letters show that you have a mismatch between local and remote character echo.
I can only assume that the default shell on mac-os (/usr/bin/sh ) is not a bash implementation like it is on Linux. Try changing the shell factory to new ProcessShellFactory("/usr/bin/bash","-i")
Sorry I don't have a mac to try this out.
Paul

Related

Vert.x ConfigRetriever.listen does not get triggered when the config file changes

I have a simple vertical that I am using to test the ConfigRetriever.listen for changes.
Using Vert.x version 4.3.4
import io.vertx.config.ConfigRetriever;
import io.vertx.config.ConfigRetrieverOptions;
import io.vertx.config.ConfigStoreOptions;
import io.vertx.core.AbstractVerticle;
import io.vertx.core.json.JsonObject;
public class MyVerticle extends AbstractVerticle {
#Override
public void start() {
ConfigStoreOptions fileStore = new ConfigStoreOptions().setType("file")
.setFormat("yaml")
.setConfig(new JsonObject().put("path", "config.yaml"));
ConfigRetrieverOptions options = new ConfigRetrieverOptions().setScanPeriod(1000)
.addStore(fileStore);
ConfigRetriever retriever = ConfigRetriever.create(vertx, options);
retriever.listen(change -> {
JsonObject previous = change.getPreviousConfiguration();
System.out.println(previous);
JsonObject changedConf = change.getNewConfiguration();
System.out.println(changedConf);
});
}
}
[Edit] The config file is under src/main/resource
When I run this, I get an output of the before as empty and after as config in my yaml file.
{}
{"bridgeservice":{"eb_address":"xyz","path":"/api/v1/aaa/","port":80}}
The problem is when I change the value in the yaml config file nothing happens. I expect the changes to get printed. When I am running this in the debugger I see
Thread [vert.x-internal-blocking-0] (Running)
..
..
..
Thread [vert.x-internal-blocking-19] (Running)
When I put the following just before the retriever.listen() , I get the succeeded... line printed and nothing from the listen method even after changing the config file values.
retriever.getConfig(ar -> {
if (ar.succeeded()) {
System.out.println("succeeded :" + ar.result());
}
else {
ar.cause()
.printStackTrace();
}
});
May be related to SO having-trouble-listen-vert-x-config-change
[Edit] The config file is under src/main/resource
When I moved my config file from resources to a folder cfg at the same level as src the Verticle behaved as it should and picked up config changes. I don't know why, maybe it's an eclipse environment thing.

automatic EOL conversion snippet?

I am working on a batch script that produces the deployment shell script according to my input, and I found my generated shell script has EOL \r\n, so I need to add a procedure to convert this file to use \n. I was looking for a solution in native batch but failed, so now I am trying to do it in other approaches.
The input is a typical shell script of commands with \r\n as EOL.
The output should be the same shell script with \n as EOL.
Any method that can be done in Windows environment will be appreciated.
nodejs approach:
const fs = require('fs');
fs.readFile('apply_patchx.sh','utf8',(err,data)=>{
fs.writeFile('apply_patch.sh',data.replace(/\r?\n/g,"\n"),()=>{
console.log('converted');
});
});
Call by node rn2n.js, input embedded in script.
Java approach:
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
public class RN2N {
public static void main(String[] args){
try {
BufferedReader br = new BufferedReader(new FileReader("apply_patchx.sh"));
BufferedWriter bw = new BufferedWriter(new FileWriter("apply_patch.sh"));
for(String line; (line = br.readLine()) != null;){
bw.write(line.replaceAll("\\\\r\\\\\n", "\\\\n"));
}
br.close();
bw.flush();
bw.close();
System.out.println("converted");
} catch (IOException e) {
System.exit(1);
}
}
}
Call by java RN2N, input embedded in script.

Windows 10 Universal Voice Commands Command Prefix

Hello I have a question regarding using Voice Commands in a Windows 10 UA. I have added a Voice Command Definition file into a Unity3D created Universal app and added the necessary code to install it at first run. However, when having started the first time, it never responds to voice commands. I added a Command Prefix, which should allow someone to start the app by uttering that prefix, but when I do that it just opens Cortana search.
I am at a loss as to why that happens.
Below are the important pieces of the code:
Xml:
<?xml version="1.0" encoding="utf-8"?>
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.2">
<CommandSet xml:lang="en-gb">
<CommandPrefix>Unity Battle</CommandPrefix>
<Example>Speak to the game</Example>
<Command Name="startWithText">
<Example>Say the message you want to appear in the game</Example>
<ListenFor> {naturalLanguage} </ListenFor>
<Feedback> Starting Game... </Feedback>
<Navigate/>
</Command>
<PhraseTopic Label="naturalLanguage" Scenario="Natural Language"/>
</CommandSet>
</VoiceCommands>
install code:
protected override async void OnLaunched(LaunchActivatedEventArgs args)
{
try
{
Windows.Storage.StorageFile vcdStorageFile =
await Package.Current.InstalledLocation.GetFileAsync(#"VoiceCommandDefinition.xml");
await
Windows.ApplicationModel.VoiceCommands.VoiceCommandDefinitionManager
.InstallCommandDefinitionsFromStorageFileAsync(vcdStorageFile);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("Installing Voice Commands Failed: " + ex.ToString());
}
}
On Activated:
protected override void OnActivated(IActivatedEventArgs args)
{
string appArgs = "";
switch (args.Kind)
{
case ActivationKind.Protocol:
ProtocolActivatedEventArgs eventArgs = args as ProtocolActivatedEventArgs;
splashScreen = eventArgs.SplashScreen;
appArgs += string.Format("Uri={0}", eventArgs.Uri.AbsoluteUri);
break;
case ActivationKind.VoiceCommand:
SpeechHelper.HandleSpeechCommand(args);
break;
}
InitializeUnity(appArgs);
}
I have already ran through the code with the debugger attached but it never hits the OnActivated(..) method.

SMS and Email Queues from Database

I just wanted to discuss a situation I am facing.
I want to send eMails to the users - a lot of eMails - but if I send them at application run time the AWS SDK is slow for emails - bad user experience - atleast for my application.
So what I plan to do is enter the data (email address, content to send, 0) in the database and launch a cron job to read the table and start sending the emails - once it sends the email - it marks the database row as 1.
I read somewhere that is a wrong practice and puts overload on the database server.
Yes, I would use intelligent crons so that no 2 crons overlap or setup a cron each for even and odd numbers etc. I am also looking at 3rd Party alternatives likes http://www.iron.io/ for crons.
Could someone share their experience with a similar situation etc. I just want to use the intelligent solution and not just put a ton of resources on the database and spend hefty on transactions...
I had to do something similar and did as Charles Engelke suggested - I used SQS.
I eliminated the database entirely by putting the entire message contents in the SQS message. You're limited to 64k in an SQS message, so as long as thats not a problem this approach is possible.
Here is sample code to queue up the message:
package com.softwareconfidence.bsp.sending;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.SendMessageRequest;
import com.googlecode.funclate.json.Json;
import java.util.HashMap;
import java.util.Map;
public class EmailQueuer {
private final AmazonSQS sqs;
private final String sendQueueUrl;
public EmailQueuer(AmazonSQS sqs,String sendQueueUrl) {
this.sqs = sqs;
this.sendQueueUrl = sendQueueUrl;
}
public void queue() {
Map<String,String> emailModel = new HashMap<String, String>(){{
put("from","me#me.com");
put("to","you#you.com");
put("cc","her#them.com");
put("subject","Greetings");
put("body","Hello World");
}};
sqs.sendMessage(new SendMessageRequest(sendQueueUrl, Json.toJson(emailModel)));
}
}
Then in your app you need to have an executor service that polls the queue and processes messages:
new ScheduledThreadPoolExecutor(1).scheduleAtFixedRate(sendEmails(), 0, 1, MINUTES)
You will need to make sure to call shutdown() on this executor when it app is exiting. Anyway, this line will send emails every minute, where sendEmails() returns an instance of this Runnable class:
package com.softwareconfidence.bsp.standalone.sending;
import com.amazonaws.services.simpleemail.AmazonSimpleEmailService;
import com.amazonaws.services.simpleemail.model.*;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.DeleteMessageRequest;
import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
import com.amazonaws.services.sqs.model.ReceiveMessageResult;
import com.amazonaws.services.sqs.model.SendMessageRequest;
import com.googlecode.funclate.json.Json;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.List;
import java.util.Map;
public class FromSqsEmailer implements Runnable {
private final AmazonSQS sqs;
private final String sendQueueUrl;
private final String deadLetterQueueUrl;
private final AmazonSimpleEmailService emailService;
public FromSqsEmailer(AmazonSimpleEmailService emailService, String deadLetterQueueUrl, String sendQueueUrl, AmazonSQS sqs) {
this.emailService = emailService;
this.deadLetterQueueUrl = deadLetterQueueUrl;
this.sendQueueUrl = sendQueueUrl;
this.sqs = sqs;
}
public void run() {
int batchSize = 10;
int numberHandled;
do {
ReceiveMessageResult receiveMessageResult =
sqs.receiveMessage(new ReceiveMessageRequest(sendQueueUrl).withMaxNumberOfMessages(batchSize));
final List<com.amazonaws.services.sqs.model.Message> toSend = receiveMessageResult.getMessages();
for (com.amazonaws.services.sqs.model.Message message : toSend) {
SendEmailResult sendResult = sendMyEmail(Json.parse(message.getBody()));
if(sendResult != null) {
sqs.deleteMessage(new DeleteMessageRequest(sendQueueUrl, message.getReceiptHandle()));
}
}
numberHandled = toSend.size();
} while (numberHandled > 0);
}
private SendEmailResult sendMyEmail(Map<String, Object> emailModel) {
Destination to = new Destination()
.withToAddresses(get("to", emailModel))
.withCcAddresses(get("cc", emailModel));
try {
return emailService.sendEmail(new SendEmailRequest(get("from", emailModel), to, body(emailModel)));
} catch (Exception e){
StringWriter stackTrace = new StringWriter();
e.printStackTrace(new PrintWriter(stackTrace));
sqs.sendMessage(new SendMessageRequest(deadLetterQueueUrl, "while sending email " + stackTrace));
}
return null;
}
private String get(String propertyName, Map<String, Object> emailModel) {
return emailModel.get(propertyName).toString();
}
private Message body(Map<String, Object> emailModel) {
Message message = new Message().withSubject(new Content(get("subject", emailModel)));
Body body = new Body().withText(new Content(get("body", emailModel)));
message.setBody(body);
return message;
}
}
One downsize of this approach if you're using a database is that the email sending step is a HTTP call. If you have a database transaction that rollsback after this HTTP call, your business process is undone, but the email is going to be sent.
Food for thought.
Thanks for the detailed response Mike. I finally ended up in implementing a REST API for my application with secure Username+Password+Key access and run it from a 3rd Party Service Iron.io which gets
www.example.com/rest/messages/format/json
It iterates and sends the messages collecting status in an array - which it then posts back to
www.example.com/rest/messagesposted
I followed this approach because I had to schedule messages for over
90 days interval and queues hold messages only for say like 14 days.
What do you recon?

When is a started service not a started service? (SQL Express)

We require programmatic access to a SQL Server Express service as part of our application. Depending on what the user is trying to do, we may have to attach a database, detach a database, back one up, etc. Sometimes the service might not be started before we attempt these operations. So we need to ensure the service is started. Here is where we are running into problems. Apparently the ServiceController.WaitForStatus(ServiceControllerStatus.Running) returns prematurely for SQL Server Express. What is really puzzling is that the master database seems to be immediately available, but not other databases. Here is a console application to demonstrate what I am talking about:
namespace ServiceTest
{
using System;
using System.Data.SqlClient;
using System.Diagnostics;
using System.ServiceProcess;
using System.Threading;
class Program
{
private static readonly ServiceController controller = new ServiceController("MSSQL$SQLEXPRESS");
private static readonly Stopwatch stopWatch = new Stopwatch();
static void Main(string[] args)
{
stopWatch.Start();
EnsureStop();
Start();
OpenAndClose("master");
EnsureStop();
Start();
OpenAndClose("AdventureWorksLT");
Console.ReadLine();
}
private static void EnsureStop()
{
Console.WriteLine("EnsureStop enter, {0:N0}", stopWatch.ElapsedMilliseconds);
if (controller.Status != ServiceControllerStatus.Stopped)
{
controller.Stop();
controller.WaitForStatus(ServiceControllerStatus.Stopped);
Thread.Sleep(5000); // really, really make sure it stopped ... this has a problem too.
}
Console.WriteLine("EnsureStop exit, {0:N0}", stopWatch.ElapsedMilliseconds);
}
private static void Start()
{
Console.WriteLine("Start enter, {0:N0}", stopWatch.ElapsedMilliseconds);
controller.Start();
controller.WaitForStatus(ServiceControllerStatus.Running);
// Thread.Sleep(5000);
Console.WriteLine("Start exit, {0:N0}", stopWatch.ElapsedMilliseconds);
}
private static void OpenAndClose(string database)
{
Console.WriteLine("OpenAndClose enter, {0:N0}", stopWatch.ElapsedMilliseconds);
var connection = new SqlConnection(string.Format(#"Data Source=.\SQLEXPRESS;initial catalog={0};integrated security=SSPI", database));
connection.Open();
connection.Close();
Console.WriteLine("OpenAndClose exit, {0:N0}", stopWatch.ElapsedMilliseconds);
}
}
}
On my machine, this will consistently fail as written. Notice that the connection to "master" has no problems; only the connection to the other database. (You can reverse the order of the connections to verify this.) If you uncomment the Thread.Sleep in the Start() method, it will work fine.
Obviously I want to avoid an arbitrary Thread.Sleep(). Besides the rank code smell, what arbitary value would I put there? The only thing we can think of is to put some dummy connections to our target database in a while loop, catching the SqlException thrown and trying again until it works. But I'm thinking there must be a more elegant solution out there to know when the service is really ready to be used. Any ideas?
EDIT: Based on feedback provided below, I added a check on the status of the database. However, it is still failing. It looks like even the state is not reliable. Here is the function I am calling before OpenAndClose(string):
private static void WaitForOnline(string database)
{
Console.WriteLine("WaitForOnline start, {0:N0}", stopWatch.ElapsedMilliseconds);
using (var connection = new SqlConnection(string.Format(#"Data Source=.\SQLEXPRESS;initial catal
using (var command = connection.CreateCommand())
{
connection.Open();
try
{
command.CommandText = "SELECT [state] FROM sys.databases WHERE [name] = #DatabaseName";
command.Parameters.AddWithValue("#DatabaseName", database);
byte databaseState = (byte)command.ExecuteScalar();
Console.WriteLine("databaseState = {0}", databaseState);
while (databaseState != OnlineState)
{
Thread.Sleep(500);
databaseState = (byte)command.ExecuteScalar();
Console.WriteLine("databaseState = {0}", databaseState);
}
}
finally
{
connection.Close();
}
}
Console.WriteLine("WaitForOnline exit, {0:N0}", stopWatch.ElapsedMilliseconds);
}
I found another discussion dealing with a similar problem. Apparently the solution is to check the sys.database_files of the database in question. But that, of course, is a chicken-and-egg problem. Any other ideas?
Service start != database start.
Service is started when the SQL Server process is running and responded to the SCM that is 'alive'. After that the server will start putting user databases online. As part of this process, it runs the recovery process on each database, to ensure transactional consistency. Recovery of a database can last anywhere from microseconds to whole days, it depends on the ammount of log to be redone and the speed of the disk(s).
After the SCM returns that the service is running, you should connect to 'master' and check your database status in sys.databases. Only when the status is ONLINE can you proceed to open it.