Efficient video game server data storage? - unity3d

I'm currently developing a multiplayer iOS video game where I want the players data to be stored in the cloud. In the game the player creates an area by placing items, planting trees etc which other players can then visit and walk around in realtime.
From the game map side what needs to be stored is:
A 50x50 tile grid - The type of tile (grass etc) and the height of
the tile
All of the placed items and any associated item specific data (A tree for example would store when it was planted)
On the player side I need to store their:
Character customisation
Inventory
The information would need to be read fairly infrequently (every time a client connects to the area) but would need to be written back to fairly regularly (every time an item is placed etc) so that any future connecting client would be reading the correct data.
My question really is where to start with regards to storing this data in an efficient way for the server? I considered using several XML files to store different portions of the map and then storing each of the other parts (Inventory, Character etc) in their own XML file. But is this the most efficient way of doing it from the servers perspective? I want to minimise the amount of data it has to write back when the client updates :)

TL;DR: Use a database and the Unity plugin for said database.
Managing an XML file via server is just going to cause a bunch of file locking/synchronization problems that you shouldn't be dealing with manually (unless, this is the focus of your project/research).
There are plenty of database backends that you can use, and that work well with Unity. A few I can think of off-hand:
• Parse.com --- they have a Unity plugin
• Write your own HTTP-based web server (in PHP/MySQL, JavaScript, Ruby, whatever) and then use a REST interface library (a bunch on the Unity Asset Store) to talk to it and store your data.

Related

Where should i store score variable?

I'm working on my multiplayer game. The game is about pvp, you can shoot your friends and all stuff like that. I decided to make a score widget, which would show who is better. You might have seen something similar to what I'm talking about in fps games like csgo, valorant, overwatch, etc. Everyone should see the score.
I tried storing variables on servers character pawn. But pawn can be destroyed and score rollbacks to 0:0.
For replicated data that is persistent to respawn, you have two main options with the built-in game framework:
Create your own subclass of AGameState and the replicated data you want in it. There is one Game state and it is replicated on all clients. See what the official wiki (bottom of the page) says about GameState. You can store game-related data in it as total kill count in team deathmatches or team capture points in domination matches.
Create your own subclass of APlayerState and the replicated data you want in it. There is one Player state per player and it is replicated on all clients. See the official API of APlayerState. You can store each players' kill count in it for example or the number of objective a player captured. Make sure data is fed from the server and replicated to clients and not the other way around.
Unreal Engine comes with a powerful Game Framework, make sure to get familiar with it.

Best Practices for serving dynamic files in a backend

does anyone know of best practices or common strategies in backend design for serving dynamic images and videos to client applications?
Background: I'm currently building an application that allows users to upload their own images and videos. I'm not really sure about how to serve these media files back to the client in the most efficient way. Do I store the files on the same VPS that my application server is running on? Do I need to save the files in different qualities / densities to better adjust for the clients' screen resolution? (I'll have mostly mobile clients)
I tried googling these questions but apparently I'm asking the wrong questions :-)
I would really appreciate maybe a reference or professional vocabulary on these topics.
Thanks in advance.
1) You need to split web server and application server.
First of all do not try to stream media files from your backend unless you can offload low-level stuff to OS - most likely you will do it wrong.
Use proxy server as an web server to serve such files.
nginx will do.
Also you need to have backup of your media files the same way as you do backup of your database.
Storing static huge media files along with application server is wrong move - it will not scale at all.
You can add cron task to move files to some CDN server - when your move is complete you replace URL in database to match new location.
So by using nginx you will save precious CPU and RAM while file is getting moved to external server.
And CDN will help you to dedicate bandwidth and CPU/RAM resources to application server.
2) Regarding image resolution and downsampling:
Screens of modern handsets have the same or even better resolution compared to typical office workstation.
Link speeds have much bigger impact on UX.
If client has smartphone with huge screen but with slow link you still have to deliver image or video as fast as possible even if quality of media will not be match the resolution of handset.
It makes sense to downsample images on demand and store result on disk for nginx/CDN to serve it again.
In case of videos it makes sense to make "bad" version with big compression(quality loss) for the cases of slow link - device will downsample it itself during playback.
And you can keep client statistics (screen sizes/downlink speeds) and generate optimized versions of such video file later when you see that it is "popular".
FYI: Several years ago some social meda giant dropped idea to prepare all possible versions of the same media file in favour of FPGA on-the-fly resampler.
I do not remember the name of the company and URL to the article. It was probably instagram.
Some cloud providers have offers with FPGA or CUDA on board to do heavy lifting.
So in some cases you could exchange storage for heave horsepower to do conversion on the fly.

WoW Addon to REST API

I´m going to create a web service for learning purposes and wanted to combine it with my WoW Hobby. My goal would be to create a "simple" Addon, which tracks my battleground activity in real time.
So when queuing for AB it enters my data in an db and when I´m out of the BG it should delete the db entry. The information should be stored in an JSON/XML-File and whenever the bg-status changes it should execute the post/update on the DB on the RESTful service.
The real time communication is very important here and I would like to know which ways of communicating to a web service are available, so I could directly dive in and create a solution.I´d like to have resources instead of solutions.
Currently I´m not used to LUA, but would like to learn it to get the knowledge of creating such a service.Which sites are you suggesting for learning LUA, especially the WoW-API?
Addons only write to disk when you log out of a character (and read that saved data when you log in) so what you intend would not be possible.*
More involved ways of communicating with the rest of the computer or even the internet are prohibited to prevent the gain of certain advantages, an example would be looking up details about your Arena opponents.
* Well, there are certainly some ways, but rather complicated ones: a program monitoring sound output to check when the BG queue pop sound is played, or a screengrabber that registers when the BG score screen comes up (which can be viewed during the match though, too)

Using CouchDB as interface. Is it appropriate way?

our devices (microscopes with cameras) produce images and additional information to each image.
Now a middleware supplies wants to connect these devices to lab automation system. They have to acquire the data and we have to provide it. An astonishing thing for me was their interface suggestion - a very cryptical token separated format (ASTM E1394-97). Unfortunatelly, they even can't accomodate images in their protocol, and are aiming to get file-paths.
I thought it is not the up-to date approach. While lookink for alternatives, I saw CoachDB.
So, my idea was, our devices would import data including images in CoachDB and they could get the data. It seems even, that using mustache, we could produce the format they want (ascii-text) and placing URLs as image references instead of path's.
My question is, did someone applied CoachDB for such a use case already? It seems to be a little-bit misuse of CoachDB, as the main intention is interface not data storage. Another point disturbing me is, that the inventor of CoachDB went to other project Coachbase. Could it mean lack of support for CoachDB in the future?
Thank you very much for any insights and suggestions!
It's ok use-case and actually we're using CouchDB in such way - as proxing middleware between medical laboratory analyzers and LIS. Some of them publish images or pdf data on shared folders and we'd just loading them into related document as attachments.
More over you'd like to know, CouchDB is able to serve external processes (aka os_daemons) and take care about their lifespan: restarting if someone had terminated and starting right after you update config options through HTTP interface. This helps to setup ASTM client and server processes since this protocol is different from HTTP (which is native for CouchDB) which communicates with devices and creates documents as regular CouchDB clients. In same way you may setup daemons to monitor shared folders for specific files. And all this is just CouchDB with few "low bounded" plugins.

Is there any value in using core data for iPhone apps?

Can people give me examples of why they would use coreData in an application?
I ask this because most apps are just clients to a central server where an API of some sort gives you the information you need.
In my case I'm writing a timesheet application for a web app which has an API and I'm debating if there is any value in replicating the data structure on my server in core data(Sqlite)
e.g
Project has many timesheets
employee has many timesheets
It seems to me that I can just connect to the API on every call for lists of projects or existing timesheets for example.
I realize for some kind of offline mode you could store locally in core data but this creates way more problems because you now have a big problem with syncing that data back to the web server when you get connection again.. e.g. the project selected for a timesheet no longer exists.
Can any experienced developer shed some light on there experiences on when core data is best practice approach?
EDIT
I realise of course there is value in storing local persistance but the key value of user defaults seems to cover most applications I can think of.
You shouldn't think of CoreData simply as an SQLite database. It's not JUST an SQLite database. Sure, SQLite is an option, but there are other options as well, such as in-memory and, as of iOS5, a whole slew of custom data stores. The biggest benefit with CoreData is persistence, obviously. But even if you are using an in-memory data store, you get the benefits of a very well structured object graph, and all of the heavy lifting with regards to pulling information out of or putting information into the data store is handled by CoreData for you, without you necessarily needing to concern yourself with what is backing that data store. Sure, today you don't care too much about persistence, so you could use an in-memory data store. What happens if tomorrow, or in a month, or a year, you decide to add a feature that would really benefit from persistence? With CoreData, you simply change or add a persistent data store, and all of your methods to get information out or in remain unchanged. The overhead for that sort of addition is minimal in comparison to if you were trying to access SQLite or some other data store directly. IMHO, that's the biggest benefit: abstraction. And, in essence, abstraction is one of the most powerful things behind OOP. Granted, building the Data Model just for in-memory storage could be overkill for your app, depending on how involved the app is. But, just as a side note, you may want to consider what is faster: Requesting information from your web service every time you want to perform some action, or requesting the information once, storing it in memory, and acting on that stored value for the remainder of the session. An in-memory data store wouldn't persistent beyond that particular session.
Additionally, with CoreData you get a lot of other great features like saving, fetching, and undo-redo.
There are basically two kinds of apps. Those that provide you with local functionality (games, professional applications, navigation systems...) and those that grant access to a remote service.
Your app seems to be in the second category. If you access remote services, your users will want to access new or real-time data (you don't want to read 2 week old Facebook posts) but in some cases, local caching makes sense (e.g. reading your mails when you're on the train with unstable network).
I assume that the value of accessing cached entries when not connected to a network is pretty low for your customers (internal or external) compared to the importance of accessing real-time-data. So local storage might be not necessary at all.
If you don't have hundreds of entries in your timetable, "normal" serialization (NSCoding-protocol) might be enough. If you only access some "dashboard-data", you will be able to get along with simple request/response-caching (NSURLCache can do a lot of things...).
Core Data does make more sense if you have complex data structures which should be synchronized with a server. This adds a lot of synchronization logic to your project as well as complexity from Core Data integration (concurrency, thread-safety, in-app-conflicts...).
If you want to create a "client"-app with a server driven user experience, local storage is not necessary at all so my suggestion is: Keep it as simple as possible unless there is a real need for offline storage.
It's ideal for if you want to store data locally on the phone.
Seriously though, if you can't see a need for it for your timesheet app, then don't worry about it and don't use it.
Solving the sync problems that you would have with an "offline" mode would be detailed in your design of your app. For example - don't allow projects to be deleted. Why would you? Wouldn't you want to go back in time and look at previous data for particular projects? Instead just have a marker on the project to show it as inactive and a date/time that it was made inactive. If the data that is being synced from the device is for that project and is before the date/time that it was marked as inactive, then it's fine to sync. Otherwise display a message and the user will have to sort it.
It depends purely on your application's design whether you need to store some data locally or not, if it is a real problem or a thin GUI client around your web service. Apart from "offline" mode the other reason to cache server data on client side might be to take traffic load from your server. Just think what does it mean for your server to send every time the whole timesheet data to the client, or just the changes. Yes, it means more implementation on both side, but in some cases it has serious advantages.
EDIT: example added
You have 1000 records per user in your timesheet application and one record is cca 1 kbyte. In this case every time a user starts your application, it has to fetch ~1Mbyte data from your server. If you cache the data locally, the server can tell you that let's say two records were updated since your last update, so you'll have to download only 2 kbyte. Now you should scale up this for several tens of thousands of user and you will immediately notice the difference of the server bandwidth and CPU usage.