I'm new to LoRaWAN. I want to set up a new gateway in my country (Algeria North Africa) since it has no gateways yet. I'm having some difficulties concerning finding the appropriate frequency and some similar problems. On the lora alliance website, I found that the suitable frequency for my region is 915MHz but when you start a new setup in thethingnetwork gateway, there is no router or frequency for my region?
How to deal with that before buying a gateway?
And for those who are from Morocco, Tunisia, Egypt, and other countries, how did you pick the suitable frequency?
Here are some images for context:
First: I've never set up a gateway so I might miss some details.
It seems to me that you know almost everything already. What I can understand with my broken french is that you should indeed be using 915MHz as the frequency as the other, 868MHz, isn't allowed in Algeria for use.
It is really sad seeing TheThingsNetwork completely forgets a continent but my solution would be to register it as if you are in the US. Because the frequency (and I'm guessing also the power limit) is equal.
And about your question for the router selection: I would suggest picking ttn-router-eu. I don't know what internet cables connect to Algeria but I'm guessing that the closest is probably a link to Europe and thus the lowest latency router would be Europe. In the grand scheme of things it probably doesn't matter much though.
Maybe you could also raise these issues on the thingsnetwork forum as the frequency/region issue might just be that no one has put in the time/effort to add it to the options. For the router, it might even be possible to start running an application server/router for the ThingsNetwork so you can serve your region, but this is speculation on my part.
Frequency Plans by Country: Algeria AS923_3
Related
This is a bit of an open ended question, but I'm looking for an open ended answer. I'm looking for a resource that can help explain how to benchmark different systems, but more importantly how to analyze the data and make intelligent choices based on the results.
In my specific case, I have a 4 server setup that includes mongo that serves as the backend for an iOS game. All servers are running Ubuntu 11.10. I've read numerous articles that make suggestions like "if CPU utilization is high, make this change." As a new-comer to backend architecture, I have no concept of what "high CPU utilization" is.
I am using Mongo's monitoring service (MMS), and I am gathering some information about it, but I don't know how to make choices or identify bottlenecks. Other servers serve requests from the game client to mongo and back, but I'm not quite sure how I should be benchmarking or logging important information from them. I'm also using Amazon's EC2 to host all of my instances, which also provides some information.
So, some questions:
What statistics are important to log on a backend setup? (CPU, RAM, etc)
What is a good way to monitor those statistics?
How do I analyze the statistics? (RAM usage is high/read requests are low, etc)
What tips should I know before trying to create a stress-test or benchmarking script for my architecture?
Again, if there is a resource that answers many of these questions, I don't need an explanation here, I was just unable to find one on my own.
If more details regarding my setup are helpful, I can provide those as well.
Thanks!
I like to think of performance testing as a mini-project that is undertaken because there is a real-world need. Start with the problem to be solved: is the concern that users will have a poor gaming experience if the response time is too slow? Or is the concern that too much money will be spent on unnecessary server hardware?
In short, what is driving the need for the performance testing? This exercise is sometimes called "establishing the problem to be solved." It is about the goal to be achieved-- because if there is not goal, why go through all the work of testing the performance? Establishing the problem to be solved will eventually drive what to measure and how to measure it.
After the problem is established, a next set is to write down what questions have to be answered to know when the goal is met. For example, if the goal is to ensure the response times are low enough to provide a good gaming experience, some questions that come to mind are:
What is the maximum response time before the gaming experience becomes unacceptably bad?
What is the maximum response time that is indistinguishable from zero? That is, if 200 ms response time feels the same to a user as a 1 ms response time, then the lower bound for response time is 200 ms.
What client hardware must be considered? For example, if the game only runs on iOS 5 devices, then testing an original iPhone is not necessary because the original iPhone cannot run iOS 5.
These are just a few question I came up with as examples. A full, thoughtful list might look a lot different.
After writing down the questions, the next step is decide what metrics will provide answers to the questions. You have probably comes across a lot metrics already: response time, transaction per second, RAM usage, CPU utilization, and so on.
After choosing some appropriate metrics, write some test scenarios. These are the plain English descriptions of the tests. For example, a test scenario might involve simulating a certain number of games simultaneously with specific devices or specific versions of iOS for a particular combination of game settings on a particular level of the game.
Once the scenarios are written, consider writing the test scripts for whatever tool is simulating the server work loads. Then run the scripts to establish a baseline for the selected metrics.
After a baseline is established, change parameters and chart the results. For example, if one of the selected metrics is CPU utilization versus the number of of TCP packets entering the server second, make a graph to find out how utilization changes as packets/second goes from 0 to 10,000.
In general, observe what happens to performance as the independent variables of the experiment are adjusted. Use this hard data to answer the questions created earlier in the process.
I did a Google search on "software performance testing methodology" and found a couple of good links:
Check out this white paper Performance Testing Methodology by Johann du Plessis
Have a look at the Methodology section of this Wikipedia article.
Does anyone know a good tool to test whether your emails are going into spam folders?
My web app generates emails to users, and I've been getting a lot of reports back from people saying "hey, no one ever responded to my message".
I have SPF rules in place and functioning correctly (email header shows an spf pass). I've also run my message through spam assassin and it scores very low.
Any other ideas?
To know if your email goes in the inbox, you need to get a metric called "Inbox Placement Rate". This indicator can be provided by Return Path, but it's quite expensive. If you're not sending huge volumes it might not worth it. The only way to measure the IPR is actually to have a certain number of test inboxes... In other words: the only way to chech that your email is not in the spam folder is to make the test and see what happen. There is not other magic solution and that's what Return Path is doing.
This means that when you hear about people claiming they have a 99% deliverability / delivery, it might be true be it just means that the email was "accepted" or "delivered" by the ISP. It's a lot, but it's not everything!
What you should do is the following: use an ESP focusing on deliverability. Personally I work for Mailjet. I believe it's the best value you can get: personalized DKIM and SPF are provided for free, you get the antispam scorings, the analytics, Ip reputation monitoring, throttling, etc. It's an all in one tool to avoid the headaches of optimizing yourself. It's more expensive that Amazon SES because you get a lot of added value services, but it has much lower prices than a lot of traditional ESPs!
Bottom line is: optimizing everything yourself is a full time job. Knowing exactly if an email is in the inbox or not will cost you a lot. The best way to proceed is to:
respect the best practices (opt in, not too much images, no red, etc.)
get some metrics such as open rates, click rates, delivery, etc. and watch their evolution over time. Any change from one sending to the other might be a signal for a problem you want to investigate.
Use a tool that takes care all the deliverability optimizations
Mailjet is cool because no matter which plan you pick, you get to use all the options. But if you want a full overview of what is existing, check out this comparison table:
http://socialcompare.com/en/comparison/transactional-emailing-providers-mailjet-sendgrid-critsend
If you're a perfectionist who wants to finetune the layout, how the emails are displayed etc. Check out Litmus, it's also a quite powerful tool!
http://litmus.com/
Simple answer: Use Mailgun!!!!
http://mailgun.net/
They will do all of your email deliverability and setup for you and give you a powerful API to build on! They are amazing. You'll never have to worry about domain keys or SPAM filtering again!
You should also check that your IP is not on any of major blacklists. dnsbl.info
This will at least give you an idea if you actually are getting flagged as spam.
For the past two years, we've used the service DeliveryMonitor.com. However, they've stopped accepting new applications which is a big red flag...
I'm currently evaluating the service from emailreach.com using their free trial
... We are now using DeliveryWatch.com with pertty good results thus far...
I am wondering if there are clusters available to rent.
Scenario:
We have a program that will take what we estimate a week to run(after optimization) on a given file. Quite possibly, longer. Unfortunately, we also need to do approximately 300+ different files, resulting in approximately 300 weeks of compute time(roundable to 6 wallclock years of continuously running job). For a research job that should be done - at the latest - by December, that's simply unacceptable. While we are exploring other options, I am investigating the option of simply renting a Beowulf cluster. The job is academic and will lead towards the completion of a PhD.
What would be ideal would be a company that we send the source and the job files to the company and then receive a week or two later the result files. Voila!
Quick googling doesn't turn up anything terribly promising.
Suggested Solutions?
Cloud computing sounds like what you need. Amazon, Microsoft and Google rent computer resources on a pay for what you use basis.
Amazon's service is the most mature, and there are several questions already about Amazon's service, EG here and here.
Amazon EC2 (Elastic Compute Cloud) sounds like exactly what you're looking for. You can sign up for one or more virtual machines (up to 20 automatically, more if you request permission), starting at $0.10 an hour per VM, plus bandwidth costs (free between EC 2 machines and Amazon's other web services). You can choose between several operating systems (various Linux distributions, OpenSolaris, Windows if you pay extra), and you can use pre-existing machine images or create your own. If you're using all open-source software and don't have much bandwidth costs, it sounds like it would cost you around $5000 to run your job (assuming that your 6 years of compute time was for something comparable to their small instances, with a single virtual CPU).
Once you sign up for the service and get their tools set up, it's pretty easy to get new virtual machines launched. I've even spent the $0.10 to launch a machine for a few minutes just to verify an answer I was giving someone here on StackOverflow; I wanted to check something on Solaris, so I just booted up an instance and had a Solaris VM at my disposal within 5 minutes.
I don't know where are you doing your PhD... Most of the Asian, European, and North American universities have some clusters. You can
meet directly the people at the lab which is in charge of the cluster.
ask your PhD director to arrange that. Maybe he/she have some friends that can handle that.
Also, the classical trick is to use the unused time of the computers of your lab/university... Basically, each computer run a client application that crunch numbers when the computer is not used. See http://boinc.berkeley.edu/
This lead may prove helpful:
http://lcic.org/vendors.html
And this is a fantastic resource site on the matter:
http://www.hpcwire.com
The thread has been replete with pointers to Amazon's EC2 - and correctly so. They are the most mature in this area. Recently, they've released their elastic map-reduce platform which sound similar (although not exactly) like what you are trying to do. Google is not an option for you as their compute model doesn't support the generic compute model you need.
For academic/scientific use, there are several public centers offering HPC capability. In Europe, there is DEISA. http://www.deisa.eu/ and DEISA members. There must be similar possibilities in the US, probably thru the NSF.
For commercial use, check IBM Deep Computing On Demand offerings.
http://www-03.ibm.com/systems/deepcomputing/cod/
There are several ways to get time on clusters.
Purchase time on Amazon elastic cloud. Depending on how familiar you are with their service, it may take time to get it configured the way you want it.
Approach a university and see if they have a commercial program to rent out the time to companies. I know several do. One that I know of specifically is private sector program at NCSA at UIUC. Depending on the institution, they also offer porting and optimization service for your code.
Or you could rent CPU time from a private provider.
I'm from Slovenia and, for example, here we have a great private provider called Arctur. The guys were helpful and and responsive when I contacted them.
You can find them here: hpc.arctur.net
One option is to rent the virtual resources equivalent of whatever number of PCs you need, and set them up as a cluster, using the Amazon Elastic Compute Cloud.
Setting up a beowulf cluster of those is entirely possible.
Check out this link which provides resources and software to do exactly that.
Go to : http://www.extremefactory.com/index.php
True HPC cluster, up to 200 TFlops.
I have landed a side project where a company wants me to develop a database / smart client to track participants in their various programs. This app will be written in C# and MS SQL Server.
I was wondering if there are any tips on how to actually bid out a price for the job?
Should I charge a flat rate (if so how do you charge it?)
Should I charge by the hour (if so what is a going rate?)
Should I enter into some sort of contract?
Any comments or tips would be very helpful, as I am completely new to doing "side gigs" outside of my normal programming job.
Thanks in advance
Yes you should enter into a contract, you need to spell out the terms and conditions of the work the expected deliverables and payment. You will also want to indicate who owns the source code.
I prefer Time & Material contracts, that is they pay an hourly rate. This allows them to change the scope without having to renegotiate the terms. If you go with a fixed bid project then as the users starts to learn what they want and change the specification, which they will, you will have to make a choice. Either you do it for free or you negotiate a scope change.
Your goal is to get the gig, and once you have it, make enough money for it to be worth your while, right? With that in mind, you can do flat rate if the client demands it, but otherwise try and get it hourly, because there will always be snags, and the client will always throw changes at you from the first day to the last minute that will suck down your time if you let it. Either way, get a contract that specifies what you are going to deliver and how to handle change requests.
I did a succession of fixed price contracts for a client once, and it worked great. Then they said they were having a little problem getting a signature on this contract from head-office, but if I were to just do the work they'd make sure I got paid for the work eventually. So I spent two months working on it, only to be told that they had been turned down by head office and so I wasn't going to get paid. I'm 90% sure they'd made a copy of my work before I could delete it from their servers, but nothing I could prove in court. So I'll never work without a signed contract again. If they don't sign a contract on day one, get the hell out of there.
If you're a beginner I'd advise against a flat rate ... unless you are prepared to shoulder the risk of more work than is paid for. It is inherently hard to estimate the effort a project takes.
The rate you can take depends on your market and how you value your time. It might be interesting accepting a low rate gig to get started in a certain market when you have free times at your hand or if you're just interested at the subject.
Well there are a few things you need to worry about.
Contract, yes, this is 100% needed, otherwise you might not get paid. The details of the contract are what might by challenging, especially about how to charge, which is what you're asking.
If you charge an hourly rate, you have to worry about being fair when you charge, and I'm sure the client will worry about this. If you're on the clock, it's not fair to check your email, YouTube, or even Stack Overflow (gasp!). Then there is also the question about whether you should charge if you need to learn some new skill. For example, if you have to learn library A, is this charged to the client or should it be done on your own time?
With a flat rate, you have to worry about keeping yourself productive so that you don't end up with a lower hourly rate. On the other hand, if the job turns out to be easy, you can finish quickly, and end up with a good hourly rate. This probably would be a lot more fair to both parties since no one would be monitoring you so that you're not goofing off and you wouldn't be doing something unethical if you decided to surf the internet while you're working.
If this is a side project and not a long-term job, I'd highly recommend a flat rate. Make sure you estimate it properly though. Take a guess of how many hours you think it will take you, add 20%, and then multiply this by some reasonable hourly rate. The other party will most likely be much more open to a flat rate since then it will be like they are paying for a finished product, rather than a developer.
Good luck!
Bidding is a tight rope walking game. Neither too low nor too much. The contract or the hourly rate always drives bidders mad. Be just in your bidding price and do not undercut too much else you might end up working for nothing. Also bid in such sites that hold the project price in escrow once the winning bidder is chosen. One good resource can be found at :- http://www.freelancing-on-net.com/writing_a_successful.php#
Even if you are bidding hourly, customers always want at least an estimate. Don't forget to account for all the activities that you don't think of when you're thinking of creating software.
Specifically, think of project management, meeting and call times with the client, end user meetings, source code management, testing, and bug fixing. In short, everything that you do for this client that prevents you from making money with another client needs to be billed to this client. You can account for that with the number of hours bid or with the hourly rate (usually a combination of the two).
I used to hear, "Prepare your best estimate. Then double it." Then I heard, "Prepare your best estimate, then times it by Pi." Now I hear, "Prepare your best estimate, then times it by Pi, then double it." :)
Unfortunately, the only way you'll get good at accurately bidding projects is to bid some, complete the project, and see where you were over or under.
You're always motivated to bid low to ensure that you get the project. But if you lose money doing a project for half your cost, getting the project might not have been a good thing.
The reason I ask is that Stack Overflow has been Slashdotted, and Redditted.
First, what kinds of effect does this have on the servers that power a website? Second, what can be done by system administrators to ensure that their sites remain up and running as best as possible?
Unfortunately, if you haven't planned for this before it happens, it's probably too late and your users will have a poor experience.
Scalability is your first immediate concern. You may start getting more hits per second than you were getting per month. Your first line of defense is good programming and design. Make sure you're not doing anything stupid like reloading data from a database multiple times per request instead of caching it. Before the spike happens, you need to do some fairly realistic load tests to see where the bottlenecks are.
For absurdly high traffic, consider the ability to switch some dynamic pages over to static pages.
Having a server architecture that can scale also helps. Shared hosts generally don't scale. A single dedicated machine generally doesn't scale. Using something like Amazon's EC2 to host can help, especially if you plan for a cluster of servers from the beginning (even if your cluster is a single computer).
You're next major concern is security. You're suddenly a much bigger target for the bad guys. Make sure you have a good security plan in place. This is something you should always have, but it become more important with high usage.
Firstly, ask if you really want to spend weeks and thousands of $ on planning for something that might not even happen, and if it does happen, lasts about 5 hours.
Easiest solution is to have a good way to switch to a page simply allowing a signup. People will sign up and you can email them when the storm has passed.
More elaborate solutions rely on being able to scale quickly. That's firstly a software issue (can you connect to a db on another server, can you do load balancing). Secondly, your hosting solution needs to support fast expansion. Amazon EC2 comes to mind, or maybe slicehost. With both services you can easily start new instances ("Let's move the database to a different server") and expand your instances ("Let's upgrade the db server to 4GB RAM").
If you keep all data in the db (including sessions), you can easily have multiple front-end servers. For the database I'd usually try a single server with the highest resources available, but only because I haven't worked with db replication and it used to be quite hard to do, at least with mysql. Things might have improved.
The app designer needs to think about scaling up (larger machines with more cores and higher performance) and/or scaling out (distributing workload across multiple systems). The IT guy needs to work out how to best support that. The network is what you look at first, because obviously everything rides on top of it. Starting at the border, that usually means network load balancers and redundant routers being served by multiple providers. You can also look at geographic caching services and apps such as cachefly.
You want to reduce your bottlenecks as much as possible. You also want to design the environment such that it can be scaled out as needed without much work. Do the design work up front and it'll mean less headaches when you do get dugg.
Some ideas (of what I used in the past and current projects):
For boosting performance (if needed) you can put a reverse-proxying, caching squid in front of your server. Of course that only works if you don't have session keys and if the pages are somewhat static (means: they change only once an hour or so) and not personalised.
With the squid you can boost a bloated and slow CMS like typo3, thus having the performance of static websites with the comfort of a CMS.
You can outsource large files to external services like Amazon S3, saving your server's bandwidth.
And if you are able to spend some (three-figures per month) bucks, you can as well use a Content Delivery Network. Whith that in place you automatically have scaling, high-availability and low latencys for your users. Of course, your pages must be cachable, so session keys and personalised pages are a no-no. If designed carefully and with CDNs in mind, you can at least cache SOME content, like pics and videos and static stuff.
The load goes up, as other answers have mentioned.
You'll also get an influx of new users/blog comments/votes from bored folks who are only really interested in vandalism. This is mostly a problem for blogs which allow completely anonymous commenting, where some dreadful stuff will be entered. The blog platform might have spam filters sufficient to block it, but manual intervention is frequently required to clean up remaining drivel.
Even a little barrier to entry, like requiring a user name or email address even if no verification is done, will dramatically reduce the volume of the vandalism.