How to send the same private message to all users in a room in an IRC channel in a rather automated way, that is, not having to do it manually for all users one by one and not having to post the message in the channel room.
To be used wisely, sure.
As you can see in the RFC 1459 that describes the IRC protocol, private messages are technically not different from channel messages. Both are the result of the PRIVMSG
command.
The PRIVMSG command allows any client to send messages to several recipients, using the following syntax:
PRIVMSG foo,bar[,anon] :Hi there for users
PRIVMSG #help,#home[,#irc] :Hi there for channels
That's supposed to work mixed-up as well.
So basically, a single target has to be either a channel or a user.
As far I read, there is no way of doing that, since every target has to be defined in the first parameter of the command.
This depends on your network. Some networks support wallmsg which works on many IRC clients as a private message to the user. This is achieved on efnet for example by sending a message to ##channel which then distributes the message to all channel ops. Equally use + to send a message to all voiced users. Unfortunately, this does not work for non-op/voice users. This works by doing PRIVMSG ##channel :Hi!
To be able to do that, in an automated fashion, you would need to write a simple script (can't help you there, you haven't stated your client) that loops over all users and then sends a PRIVMSG to them one by one or in the mass message mode where you separate the users by commas (and no spaces) like PRIVMSG user1,user2,user3 :message.
Most likely you want the latter option. Be careful with your flood thresholds though!
You could add a line to your mirc popups nick list
...message all selected {%i = 0 | %nicks = "" | :nextnick | inc %i | if ($snick(#,%i) == $null) { if ($len(%nicks) > 0) /msg %nicks $$?="message " | halt } | %nicks = %nicks $snick(#,%i) | if (4 // %i) { /msg %nicks $$?="message " | %nicks = "" } | goto nextnick | }
i use this on my network along with op all and deop all or kick all
Related
Trying to figure out how to send/read data through socket. On remote server I create new netcat -l 4444 and from local send text data echo "test" | netcat remote.host 4444. This is always works fine.
Trying to reproduce:
require "socket"
HOST = "remote.host"
PORT = 4444
ch_request = Channel(String).new
ch_response = Channel(String).new
spawn do
socket = TCPSocket.new(HOST, PORT)
loop do
select
when request = ch_request.receive
socket << request
socket.flush
end
if response = socket.gets
ch_response.send response
end
end
end
sleep 0.1
ch_request.send "hello"
loop do
select
when response = ch_response.receive
pp response
end
end
In my dream I send data to channel, read it from first loop then send to socket. The same way but reverse order need for read it from second loop.
On practice this is not happens. On local after connect I got "test" and can't send back anything. On remote I can send to local but on local got only empty string once and nothing more after.
What mean this behavior and how to achieve planned?
You didn't show this, but I suppose you have a second implementation using TCPServer for the netcat -l equivalent.
You need to use separate fibers for reading/writing to the socket and the channel. It's a bit hard to judge what exactly happens without seeing the server, but I suppose you end up in a deadlock where both sides are waiting on input of the other side or user, but cannot proceed to actually send or read anything. In other words you interlocked the sending/receiving part, requiring the other side to carefully react and interplay so to not lock up the client. This is obviously a brittle approach.
Instead you should make sure any fiber does not do more than one operation in a loop. One receives from the socket and forwards that to a channel, the second one receives from a channel and forwards that to the socket, the third one receives from the reader side channel and prints or does whatever you want to do the data and the last one fills the sender channel. This way no operation can block one of the others. Of course one of those fibers should simply be the main program one.
In the server you additionally need one fiber that accepts the client connections and spawns the sender and receiver loops for each.
Finally note that a select statement with a single when branch has no effect, you can make the call directly. select is useful if you need to read or write to multiple channels concurrently in the same fiber, so for example if you would have multiple channels providing data to be send out to a socket, you would use select to not have the messages be corrupted by two fibers writing to the same socket at the same time. An additional usecase for select is to send or receive from a channel with a timeout.
Who is looking for an answer to similar questions. The final result what I wanted looks like this:
# Open socket for simulate remote server: `netcat -v -4 -l 4444`
require "socket"
HOST = "remote.host"
PORT = 4444
# JFYI: In real life this packed into class and I use class variable instead consts.
TUBE_REQUEST = Channel(String).new
TUBE_RESPONSE = Channel(String).new
SOCKET = TCPSocket.new(HOST, PORT)
spawn do
until SOCKET.closed?
if request = TUBE_REQUEST.receive
SOCKET << request
SOCKET.flush
end
end
end
spawn do
until SOCKET.closed?
if response = SOCKET.gets
TUBE_RESPONSE.send response
end
end
end
sleep 0.1
def receive_response
TUBE_RESPONSE.receive
end
def send(message, wait_for_response = true)
TUBE_REQUEST.send message
receive_response if wait_for_response
end
send("command with response")
send("command with new line and response\n")
send("command without new line and response", false)
It will send each command and wait for answer (except the last) from remote and then call the next command.
Background
We're building an application that will process & send emails via Mailgun. These are sometimes one-off messages, initiated by a transaction. Some emails, though, will be sent to 30k+ at once.
Eg, a newsletter to all members.
Considerations
Mailgun offers a Batch Sending option with their API. Using "Recipient Variables", you can include dynamic values that are paired with a particular user.
This Batch Sending functionality is limited, however. You cannot send more than 1,000 recipients per request, which means we have to iterate through a recipient list (on our database) for each set of 1,000. Mailgun provides an example of how this might work, using Python (scroll about 2/3 down).
Question
Are there any advantages to batch sending (ie, sending an email to a group of recipients through a single API call, using recipient variables) as opposed to making our own loop, variable substitutions and individual API calls?
I assume this is more taxing on our server, as it would be processing each message itself, instead of just offloading all that data to Mailgun's server for heavy-lifting on their end. But I also like the flexibility & simplicity of handling that on our end and sending a "fully-rendered" message to Mailgun, one at a time, without having to iterate 1k at a time.
Any thoughts on best practices, or considerations we should take into account?
Stumbled onto this today, and felt it provided a pretty good summary/answer for my original question. I wanted to post this as an answer, in case anybody else has this question and hasn't found this Mailgun post. Straight from the horse's mouth, too. The nutshell version:
For PHP, at least, the SDK has a Mailgun class, with a BatchMessage() method. This actually handles the counting of recipients for you, so you can just queue up as many email addresses as you want (ie, more than 1k) and Mailgun will fire off to the API endpoint as needed. Pretty slick!
Here's their original wording, plus a link to the page.
Sending a message with Mailgun PHP SDK + Batch Message:
Batch Message
In addition to Message Builder, we have Batch Message. This class
allows you to build a message and submit a template message in
batches, up to 1,000 recipients per post. The benefit of using this
class is that the recipients tally is monitored and will automatically
submit the message to the endpoint when you've added the 1,000th
recipient. This means you can build your message and begin iterating
through your database. Forget about sending the message, the SDK will
keep track of posting to the API when necessary.
// First, instantiate the SDK with your API credentials and define your domain.
$mgClient = new Mailgun("key-example");
$domain = "example.com";
// Next, instantiate a Message Builder object from the SDK, pass in your sending domain.
$batchMsg = $mgClient->BatchMessage($domain);
// Define the from address.
$batchMsg->setFromAddress("dwight#example.com",
array("first"=>"Dwight", "last" => "Schrute"));
// Define the subject.
$batchMsg->setSubject("Help!");
// Define the body of the message.
$batchMsg->setTextBody("The printer is on fire!");
// Next, let's add a few recipients to the batch job.
$batchMsg->addToRecipient("pam#example.com",
array("first" => "pam", "last" => "Beesly"));
$batchMsg->addToRecipient("jim#example.com",
array("first" => "Jim", "last" => "Halpert"));
$batchMsg->addToRecipient("andy#example.com",
array("first" => "Andy", "last" => "Bernard"));
// ...etc...etc...
// After 1,000 recipeints,
// Batch Message will automatically post your message to the messages endpoint.
// Call finalize() to send any remaining recipients still in the buffer.
$batchMsg->finalize();
The answer of #cdwyer and #nikoshr is very helpful, but bit legacy. Used methods in the example are deprecated. Here is current usage of lib:
$batchMessage = $this->mailgun->messages()->getBatchMessage('mydomain.com');
$batchMessage->setFromAddress('user#domain.com');
$batchMessage->setReplyToAddress('user2#domain.com');
$batchMessage->setSubject('Contact form | Company');
$batchMessage->setHtmlBody('<html>...</html>');
foreach ($recipients as $recipient) {
$batchMessage->addToRecipient($recipient);
}
$batchMessage->finalize();
More info at documentation.
I want to extract all e-mail addresses from GMAIL and GROUP and SORT by the email address. The result is a sorted list of email addresses I contact the most.
After some Googling
I have tried to export contacts (there is an option to select most contacted) - but this resulted in 20 most contacted. Does seem Gmail is counting ...
There is a script by labnol that exports ALL e-mail adresses ... but this is rather slow & does not do the count
question: How can I extract all e-mail addresses from GMAIL and GROUP and SORT by the email address?
Would an export to IMAP work (and count from there)? or is there a smarter way
Many thanks
You have two options
You can indeed extract headers (just headers, you don't need the full message body) from IMAP.
You can use the Gmail API. If you want to count who sends you messages, something like this should work.
Sample code using Gmail API in Python
# Retrieve a page of threads
threads = gmail_service.users().threads().list(userId='me',fields='threads(id)').execute()
# Print ID for each thread
pp = pprint.PrettyPrinter(depth=5)
if threads['threads']:
for thread in threads['threads']:
print 'Thread ID: %s has messages from senders:' % (thread['id'])
t = gmail_service.users().threads().get(userId='me',id=thread['id'],fields='messages/payload/headers').execute()
for msg in t['messages']:
for header_from in [v for v in msg['payload']['headers'] if v['name'] == 'From']:
# There should be only one, but sometimes From is missing
from_field_val = header_from['value']
print from_field_val
# TODO Extract email address and increment count
https://gist.github.com/regisd/3b4733e974483d7994fe
I noticed in the FAQ, in the Monitoring section, that it's not possible to get a list of connected peers or to be notified when peers connect/disconnect.
Does this imply that it's also not possible to know which topics a PUB/XPUB socket knows it should publish, from its upstream feedback? Or is there some way to access that data?
I know that ZMQ >= 3.0 "supports PUB/SUB filtering at the publisher", but what I really want is to filter at my application code, using the knowledge ZMQ has about which topics are subscribed to.
My use-case is that I want to publish info about the status of a robot. Some topics involve major hardware actions, like switching the select lines on an ADC to read IR values.
I have a publisher thread running on the bot that should only do that "read" to get IR data when there are actually subscribers. However, since I can only feed a string into my pub_sock.send, I always have to do the costly operation, even if ZMQ is about to drop that message when there are no subscribers.
I have an implementation that uses a backchannel REQ/REP socket to send topic information, which my app can check in its publish loop, thereby only collecting data that needs to be collected. This seems very inelegant though, since ZMQ must already have the data I need, as evidenced by its filtering at the publisher.
I noticed that in this mailing list message, the OP seems to be able to see subscribe messages being sent to an XPUB socket.
However, there's no mention of how they did that, and I'm not seeing any such ability in the docs (still looking). Maybe they were just using Wireshark (to see upstream subscribe messages to an XPUB socket).
Using zmq.XPUB socket type, there is a way to detect new and leaving subscribers. The following code sample shows how:
# Publisher side
import zmq
ctx = zmq.Context.instance()
xpub_socket = ctx.socket(zmq.XPUB)
xpub_socket.bind("tcp://*:%d" % port_nr)
poller = zmq.Poller()
poller.register(xpub_socket)
events = dict(poller.poll(1000))
if xpub_socket in events:
msg = xpub_socket.recv()
if msg[0] == b'\x01':
topic = msg[1:]
print "Topic '%s': new subscriber" % topic
elif msg[0] == b'\x00':
topic = msg[1:]
print "Topic '%s': subscriber left" % topic
Note that the zmq.XSUB socket type does not subscribe in the same manner as the "normal" zmq.SUB. Code sample:
# Subscriber side
import zmq
ctx = zmq.Context.instance()
# Subscribing of zmq.SUB socket
sub_socket = ctx.socket(zmq.SUB)
sub_socket.setsockopt(zmq.SUBSCRIBE, "sometopic") # OK
sub_socket.connect("tcp://localhost:%d" % port_nr)
# Subscribing zmq.XSUB socket
xsub_socket = ctx.socket(zmq.XSUB)
xsub_socket.connect("tcp://localhost:%d" % port_nr)
# xsub_socket.setsockopt(zmq.SUBSCRIBE, "sometopic") # NOK, raises zmq.error.ZMQError: Invalid argument
xsub_socket.send_multipart([b'\x01', b'sometopic']) # OK, triggers the subscribe event on the publisher
I'd also like to point out the zmq.XPUB_VERBOSE socket option. If set, all subscription events are received on the socket. If not set, duplicate subscriptions are filtered. See also the following post: ZMQ: No subscription message on XPUB socket for multiple subscribers (Last Value Caching pattern)
At least for the XPUB/XSUB socket case you can save a subscription state by forwarding and handling the packages manually:
context = zmq.Context()
xsub_socket = context.socket(zmq.XSUB)
xsub_socket.bind('tcp://*:10000')
xpub_socket = context.socket(zmq.XPUB)
xpub_socket.bind('tcp://*:10001')
poller = zmq.Poller()
poller.register(xpub_socket, zmq.POLLIN)
poller.register(xsub_socket, zmq.POLLIN)
while True:
try:
events = dict(poller.poll(1000))
except KeyboardInterrupt:
break
if xpub_socket in events:
message = xpub_socket.recv_multipart()
# HERE goes some subscription handle code which inspects
# message
xsub_socket.send_multipart(message)
if xsub_socket in events:
message = xsub_socket.recv_multipart()
xpub_socket.send_multipart(message)
(this is Python code but I guess C/C++ looks quite similar)
I'm currently working on this topic and I will add more information as soon as possible.
I have a bot that replies to users. But sometimes when my bot sends its reply, the user or their email provider will auto-respond (vacation message, bounce message, error from mailer-daemon, etc). That is then a new message from the user (so my bot thinks) that it in turn replies to. Mail loop!
I'd like my bot to only reply to real emails from real humans. I'm currently filtering out email that admits to being bulk precedence or from a mailing list or has the Auto-Submitted header equal to "auto-replied" or "auto-generated" (see code below). But I imagine there's a more comprehensive or standard way to deal with this. (I'm happy to see solutions in other languages besides Perl.)
NB: Remember to have your own bot declare that it is autoresponding! Include
Auto-Submitted: auto-reply
in the header of your bot's email.
My original code for avoiding mail loops follows. Only reply if realmail returns true.
sub realmail {
my($email) = #_;
$email =~ /\nSubject\:\s*([^\n]*)\n/s;
my $subject = $1;
$email =~ /\nPrecedence\:\s*([^\n]*)\n/s;
my $precedence = $1;
$email =~ /\nAuto-Submitted\:\s*([^\n]*)\n/s;
my $autosub = $1;
return !($precedence =~ /bulk|list|junk/i ||
$autosub =~ /(auto\-replied|auto\-generated)/i ||
$subject =~ /^undelivered mail returned to sender$/i
);
}
(The Subject check is surely unnecessary; I just added these checks one at a time as problems arose and the above now seems to work so I don't want to touch it unless there's something definitively better.)
RFC 3834 provides some guidance for what you should do, but here are some concrete guidelines:
Set your envelope sender to a different email address than your auto-responder so bounces don't feed back into the system.
I always store in a database a key of when an email response was sent from a specific address to another address. Under no circumstance will I ever respond to the same address more than once in a 10 minute period. This alone stopped all loops, but doesn't ensure nice behavior (auto-responses to mailing lists are annoying).
Make sure you add any permutation of header that other people are matching on to stop loops. Here's the list I use:
X-Loop: autoresponder
Auto-Submitted: auto-replied
Precedence: bulk (autoreply)
Here are some header regex's I use to avoid loops and to try to play nice:
/^precedence:\s+(?:bulk|list|junk)/i
/^X-(?:Loop|Mailing-List|BeenThere|Mailman)/i
/^List-/i
/^Auto-Submitted:/i
/^Resent-/i
I also avoid responding if any of these are the envelop senders:
if ($sender eq ""
|| $sender =~ /^(?:request|owner|admin|bounce|bounces)-|-(?:request|owner|admin|bounce|bounces)\#|^(?:mailer-daemon|postmaster|daemon|majordomo|ma
ilman|bounce)\#|(?:listserv|listsrv)/i) {
That really sounds like something that's probably available as a module from CPAN, but I didn't find anything clearly relevant in five minutes of searching. Mail::Lite::Mbox::Processor looks like it might do what you want:
Mail::Lite::Message::Matcher is a
framework for automated mail
processing. For example you have a
mail server and you have a need to
process some types of incoming mail
messages automatically. For example,
you can extract automated
notifications, invoices, alerts etc.
from your mail flow and perform some
tasks based on content of those
messages.
but its docs are sparse enough that it isn't immediately obvious whether it provides those example functions itself or if you have to provide the code to drive them.
In any case, though, if you haven't already checked CPAN, that's where I would start if I wanted to do something like this.
My answer here only deals with bounces which is more straightforward.
Using DSN (Delivery Status Notification) identifier will help you detect a DSN/bounced message. It should go to Return-Path and not Reply-To.
Here's a sample of a typical DSN message. The header information includes the message id, content type has specific values (delivery-status) etc.
Not able to provide you any codes in perl, just my 2 cents of idea.
PS: Do note that not all mail servers or MTA conforms to this, but I guess most do.
There should be a standard way of dealing with this, but the problem is that you'd have to assume that systems that send auto-replies comply to that standard, when most the time, they just don't.
How do you get the address that you reply to? I hope you aren't using the From: header. Check the Reply-to: header first and if that doesn't exist, use the Return-path:.
But whatever you do, you will simply have to keep a log of what you sent to whom and throttle your bot to some sensible value of messages per time.