We have a range of web applications here that allow users to download selected data from a number of databases and online services. Mainly Environmental information. We can track users visiting web pages using tools like Piwik or Google Analytics. We also want to track the amount of resource or data that they use, possibly also applying limits to record downloads.
If this was a single DB system we could track rows delivered within the db. However here we have a SOA with a range of sources and sinks. What I envisage is a service that can be messaged by other systems to register or track the amount of a resource used.
e.g User Andrew was sent 125MB of water quality data.
The central data metering service tracks usage messages from a variety of sources, produces reports and where appropriate applies caps or billing limits.
This service might be expanded to include processing as well as data download.
I would consider this to be a not unusual requirement but I can't find much in the way of existing software for it - perhaps because I am not using the correct terminology.
SO my questions:
What would you call this service - what keywords will lead me to existing systems?
What solutions already exist in this area - in particular FOSS or cloud based systems?
Could something like Google Analytics be persuaded to operate in this fashion?
It would be possible to do with the measurement protocol from Google Universal Analytics in conjunction with the user id feature in Analytics and one or more custom dimensions.
The measurement protocol is a language agnostic vaguely REST-like (inasfar as you send a bunch of parameters to an endpoint) protocol to send tracking data to the Google servers.
User id is a feature to recognize authenticated users across devices and multiple visits.
If the various parts of your setup send http calls build to the measurement protocol and include the user id to recognize the user and a value for a custom dimension for the file size (or rather a custom metric if you want to have sums and averages) and maybe a custom dimension for the file name you can send this to you Analytics account and build a custom report for downloads.
Note that the user id is an internal id that is used to link together visits by the same user from multiple devices - it is not something that shows up in the reports that would allow you to report on individual users in the Analytics interface (if you want that you need to include another id as custom dimension, and you have to check with the Google TOS what kind of id is allowed). Plus you'd need a dedicated data view in GA for sessions with a user id which will not show unauthenicated users.
I'm writing a piece of software that will run on computers as well as phones.
The service uses an HTTP API for communication and will be published over the local network using Zeroconf.
Initially I published my service using _http._tcp. as the service type but I quickly discovered that both my NAS and my music receiver(!) also broadcasts themselves with that exact service type.
So the question now arises how to differentiate between my service and other services that are using HTTP.
Alternatives
Using a different service type
The is certainly the most certainly the easiest way and (almost) guarantees no other services will be picked up.
However, according to Apple1 new services should be registered with IANA. This is obviously not required but seeing as they recommend it it feels like it would be the wrong way to do it
Using the TXT record
Apple2 describes the TXT record like this:
When a service is registered, three related DNS records are created: a service (SRV) record, a pointer (PTR) record, and a text (TXT) record. The TXT record contains additional data needed to resolve or use the service, although it is also often empty.
The certainly feels like it could be the right way to do it, but I'm still not sure and it's hard to find a description of what the field should contain.
My first though would be to put something like <service_name>-<version> which will then be parsed to see which service it actually is.
My NAS seems to use this for identifying model and version numbers.
Try talking to the service
After finding a service one could always perform a HEAD request on a known endpoint and look for a known header set by the service.
This feels like a fairly slow approach and who knows what making a HEAD request to my receiver will do.
And just to be clear, this question has nothing to do with a specific language or framework, it's about the concepts of zeroconf.
I could show some code but I don't see how that would help.
First, does the service you're advertising actually meet the qualifications for _http as defined by RFC 2782. Specifically- is it not just using HTTP for a transport but is also:
can be displayed by "typical" web browser client software, and
is intended primarily to be viewed by a human user.
If no, register your own service type (there are a couple other services that use HTTP as a transport but don't meet those qualifications so they have -http as a suffix to the service name, see pgpkey-http, senteo-http, xul-http).
If yes, there are a couple ways to go depending on how strict one's interpretation of the RFC is. The least strict being just adding a TXT record as you've already noted in your question. iTunes registers itself with a TXT record in the format iTSh Version=196618.
If you're feeling a little more strict, the RFC only explicitly states that the u=, p= and path= TXT records exist for HTTP. Perhaps someone can chime in on this, but I haven't seen much discussion on whether adding TXT records to already existing entries is frowned upon or not. So with that, the other way is to just an algorithmic instance name. For example, adding the suffix "-NicklasAService" to the device name. Hopefully giving it a unique name to the local network but still making it so that the service can be easily picked out by the PTR record by just looking for the suffix.
I'm about to deploy my first production version of a web service that uses MongoDB.
This web service could be prone to attacks (hackers).
I have been using the built-in Object-ID as unique identifier for each value, and this is exposed publicly (or at least to authenticated users).
Could this be a problem considering that it's built up on data such as, object creation timestamp, machine and process IDs etc (http://docs.mongodb.org/manual/core/object-id/)?
Could it be that I'm giving away too much information about when the object was created, how many machines that is used to etc?
What would your recommendations be?
Not really. It can't be used to graph your network or your machines or find hidden objects or to even graph your traffic and load times (as I have found out myself from trying).
For example, unlike an auto incrementing ID you cannot easily judge what time stamp or PID or Machine ID was used to create the next ObjectID. This makes it very hard to crawl for the hidden objects especially if you don't publicly link to them somewhere.
The PID and Machine ID are not very good at identifying anything about your network really and the PID can change almost any time it likes, whether the process was restarted or whether you restarted the server; or if you are using a language like PHP, every time a new connection comes in.
The machine ID is another piece of useless information that doesn't really give any meaningful results for anyone but your computer. I don't believe it uses the network interfaces ID (some drivers did) any more so it cannot be used to identify the machine externally.
So in short, not really.
I have a site that has different features and functionality based on the location that the user has selected. Each location has events that are unique to that location. These events are the primary reason that a user comes to the site for.
There are key areas of the site that are not navigable unless a location is selected for the user. Therefore, the location has to be presumed until the selected location is known -- I plan on using some basic GeoIP lookups for this.
So the requirements basically boil down to:
Location must be set before any page on the site is loaded
User can change the location setting
The location setting must persist from request to request
This setting should be accessible to the layout and view and will be a prerequisite for most controllers
I can think of a few ways off the top of my head to tackle this (Zend_Registry, custom Zend_Session_Namespace, etc) but I'm curious if there isn't a more widely identified strategy for this type of problem.
I see this as similar to setting language for a site with i18n and so i would implement in a similar fashion. Id probably store the location in a session value, or if the user is registered they could obviously save it to their settings/profile in whatever backend storage mechanism is being used. You could also add this location to the route params as is often done with i18n but that could get messy given its a geo location and not just a locale.
I don't think Zend_Registry might fit your needs, since it will only make location settings available throughout the application during a single request but will not persist.
If you need these settings to be persistent between requests, your best (and very likely - only) choice is Zend_Session.
As to #1, I suggest using a controller plugin for that.
Let's say I need to access a web service from an iPhone app. This web service requires clients to digitally sign HTTP requests in order to prove that the app "knows" a shared secret; a client key. The request signature is stored in a HTTP header and the request is simply sent over HTTP (not HTTPS).
This key must stay secret at all times yet needs to be used by the iPhone app.
So, how would you securely store this key given that you've always been told to never store anything sensitive on the client side?
The average user (99% of users) will happily just use the application. There will be somebody (an enemy?) who wants that secret client key so as to do the service or client key owner harm by way of impersonation. Such a person might jailbreak their phone, get access to the binary, run 'strings' or a hex editor and poke around. Thus, just storing the key in the source code is a terrible idea.
Another idea is storing the key in code not a string literal but in a NSMutableArray that's created from byte literals.
One can use the Keychain but since an iPhone app never has to supply a password to store things in the Keychain, I'm wary that someone with access to the app's sandbox can and will be able to simply look at or trivially decode items therein.
EDIT - so I read this about the Keychain: "In iPhone OS, an application always has access to its own keychain items and does not have access to any other application’s items. The system generates its own password for the keychain, and stores the key on the device in such a way that it is not accessible to any application."
So perhaps this is the best place to store the key.... If so, how do I ship with the key pre-entered into the app's keychain? Is that possible? Else, how could you add the key on first launch without the key being in the source code? Hmm..
EDIT - Filed bug report # 6584858 at http://bugreport.apple.com
Thanks.
The goal is, ultimately, restrict access of the web service to authorized users, right? Very easy if you control the web service (if you don't -- wrap it in a web service which you do control).
1) Create a public/private key pair. The private key goes on the web service server, which is put in a dungeon and guarded by a dragon. The public key goes on the phone. If someone is able to read the public key, this is not a problem.
2) Have each copy of the application generate a unique identifier. How you do this is up to you. For example, you could build it into the executable on download (is this possible for iPhone apps)? You could use the phone's GUID, assuming they have a way of calculating one. You could also redo this per session if you really wanted.
3) Use the public key to encrypt "My unique identifier is $FOO and I approved this message". Submit that with every request to the web service.
4) The web service decrypts each request, bouncing any which don't contain a valid identifier. You can do as much or as little work as you want here: keep a whitelist/blacklist, monitor usage on a per-identifier basis and investigate suspicious behavior, etc.
5) Since the unique identifier now never gets sent over the wire, the only way to compromise it is to have physical access to the phone. If they have physical access to the phone, you lose control of any data anywhere on the phone. Always. Can't be helped. That is why we built the system such that compromising one phone never compromises more than one account.
6) Build business processes to accommodate the need to a) remove access from a user who is abusing it and b) restore access to a user whose phone has been physically compromised (this is going to be very, very infrequent unless the user is the adversary).
The simple answer is that as things stand today it's just not possible to keep secrets on the iPhone. A jailbroken iPhone is just a general-purpose computer that fits in your hand. There's no trusted platform hardware that you can access. The user can spoof anything you can imagine using to uniquely identify a given device. The user can inject code into your process to do things like inspect the keychain. (Search for MobileSubstrate to see what I mean.) Sorry, you're screwed.
One ray of light in this situation is in app purchase receipts. If you sell an item in your app using in app purchase you get a receipt that's crypto signed and can be verified with Apple on demand. Even though you can't keep the receipt secret it can be traced (by Apple, not you) to a specific purchase, which might discourage pirates from sharing them. You can also throttle access to your server on a per-receipt basis to prevent your server resources from being drained by pirates.
UAObfuscatedString could be a solution to your problem. From the docs:
When you write code that has a string constant in it, this string is saved in the binary in clear text. A hacker could potentially discover exploits or change the string to affect your app's behavior. UAObfuscatedString only ever stores single characters in the binary, then combines them at runtime to produce your string. It is highly unlikely that these single letters will be discoverable in the binary as they will be interjected at random places in the compiled code. Thus, they appear to be randomized code to anyone trying to extract strings.
If you can bear to be iPhone OS 3.0-only, you may want to look at push notifications. I can't go into the specifics, but you can deliver a payload to Apple's servers along with the notification itself. When they accept the alert (or if your app is running), then some part of your code is called and the keychain item is stored. At this point, that is the only route to securely storing a secret on an iPhone that I can think of.
I had the same question and spent a lot of time poking around for an answer. The issue is a chicken and egg one: how to pre-poluate the keychain with data needed by your app.
In any case, I found a technique that at least will make it harder for a jailbreaker to uncover the information - they'll at least have to disassemble your code to find out what you did to mask the info:
String Obfuscation (if the link breaks search for "Obfuscate / Encrypt a String (NSString)")
Essentially the string is obfuscated before placed in the app, then you unobfuscate it using code.
Its better than doing nothing.
David
EDIT: I actually used this in an app. I put a base coding string into the info.plist, then did several operations on it in code - rot13, rotate/invert bytes, etc. The final processed string was used to decode the obfuscated string. Now, the three letter agencies could for sure break this - but at a huge cost of many hours decoding the binary.
I was going to say that this is the best technique I've come across, but I just read Kiran's post on UAObfuscatedString (different answer), which is a completely different way to obfuscate. It has the benefit of no strings saved anywhere in the app - each letter is turned into a method call. The selectors will show up as strings, so a hacker can quickly tell that your class used that technique though.
I think that this similar question, and my answer, may be relevant to your case too. In a nutshell, there was some talk of a trusted platform module being present in an iPhone. This would allow your service to trust an iPhone, even in the hands of an attacker. However, it looks like using the keychain is your best bet.
Did you consider/try the Push Notification suggestion, for initially transmitting the secret to the app & keychain? Or end up finding some other method to achieve this?
I'm going have my iphone app upload images to Amazon S3. Instead of putting the AWS credentials in the app, I am going to have the app phone home to my server for the URI and headers to use in the S3 upload request. My server will generate the S3 URI, proper signatures, etc. I can then implement a tighter, more specific security model on my app's webservice than AWS offers by itself and not give away my AWS keys to anyone with a jailbroken iphone.
But there still has to be some trust (credentials or otherwise) given to the app, and that trust can be stolen. All you can ever do is limit the damage done if someone jailbreaks an iphone and steals whatever credentials are in the app. The more powerful those credentials are, the worst things are. Ways to limit the power of credentials include:
avoid global credentials. make them per-user/application
avoid permanent credentials. make them temporary if possible
avoid global permissions. give them only the permissions they need. for instance, write permissions might be broken down into insert, overwrite, delete, write against resource group A or B, etc, and read could be broken into read named resources, read a list of all existing resources, read resource groups A or B, etc.
I would recommend creating a key at run time if possible. This way if the key were to get apprehended during a particular session, once the session ends, the key will be worthless. They could still apprehend the key from memory if they are smart enough, but it wouldn't matter since the key would become invalid after a period of time.
Sounds wonky. Would use HTTPS and maybe an encryption package to handle the key.
I think CommonCrypto is available for iPhone.
EDIT: Still sounds wonky. Why would anyone pass a secret key in an HTTP header? Anyone who traces your network traffic (via a logging wifi router, for instance) would see it.
There are well-established security methods for encrypting message traffic...why not use them rather than invent what is basically a trivially flawed system?
EDIT II: Ah, I see. I would go ahead and use the Keychain...I think it is intended for just these kinds of cases. I missed that you were generating the request using the key. Would still use HTTPS if I could though, since that way you don't risk people deducing your keygeneration scheme via inspection of enough signatures.