Related
At our company, we are looking at replacing a number of legacy systems that handle information from our customers into our company. Typical systems allow the user to drop an ftp file somewhere. This file will then be transformed by a number of programs and eventually end up in some kind of database. In total we have +30 different "systems" or applications that does this. And, it is more or less a mess.
We believe we lack a common system to manage these flows: triggered by upload or possible another event, register the data, create some sort of "job" (or process) from it, pass it through the variuos services/transformation programs it needs to go through, provide feedback to the customer, provide information about progress, etc to us, handle failures and so on. Sort of like Jenkins (/Hudson/CruiseControl/similar) but for information transformation jobs, rather than build jobs, and with a job beeing more of a "process instance" of a job, then the job itself (e.g. different data should trigger the job several times, running concurrently).
We are cabable of writing such software ourselves, but surely software as this exists(?) I have been googling around, and found that what we need ma possibly be "job scheduling" software or "business process management" software. However, these are all new domains for us, and I am quite uncertain to as what kind of software would fit our needs. It appears one could invest quite a deal of ressources into this type of software before
So, what I am looking for is pointers to what kind of software or systems that could solve the kind of needs we have. Preferably Open Source, Java based, running in a Java EE container or similar, but really, at this point, almost any pointer/hint will be welcomed :-)
Thanks in advance
P.S. I realise I may be out of scope for Stackexchange, but I have been unable to locate another forum where this kind of question might be answered, so I hope it is OK.
I know of the following products:
Redwood Cronacle (I worked with it 1994-1997 and it still runs). Purchase product. Oracle and C based. Strong in multiple server platforms. Embeddable.
Oracle E-business suite core. Purchase product. Oracle based. Strong for integration with the same ERP system. Weak for multiple server platforms.
Invantive Vision (I developed it :-). Purchase product. Oracle and Java based. Strong in integration with ETL (Pentaho open source). Weak for multiple server platforms. Embeddable.
Quartz Scheduler. Apache license. Java based. Worked with in 2004 or so. Strong focus on embedding.
Hi I don’t know if you will find that solution in open source or Java. It sounds like bespoke or custom software to me. I would advise you to search for a project management software developer with high level of IT and Data warehousing. Ask for bespoke and customized installations with a real time database. I think you will solve your problem with this.
Me and few friends run a little app creation business in our spare time, our current development environment is a 3 macbooks laptops running just snow leopard, 4 asus laptops with dual boot windows 7 and ubuntu and a rubbish test server box that is similar to our vps.
Our setup currently work okayish at the moment, with a few minor issues, like not knowing what version of software we are working on, caused by continually switching operating systems and lost of productivity from being to lazy to switch the laptop we are working, having to unplug it and plug in the new one, including the second monitor, keyboard and mouse.
Our system is far from professional and we are looking to upgrade. This is because we wish to increase our staff and we have some cash saved up, so why not. The phone we are targetting are iOS, android and Win7. Our servers are written in php and json. So my question is basically, how do you guys manage with all these multiple operating systems.
iOS requires mac os x
android can use all
json require linux/mac os x
windows phone 7 requires windows
do you guys use some form of virutalization?
or try those libraries that compile to each phone binary such as unity?
There are many many different ways to solve this and you may have to find what works best for you. Here are some suggestions though.
Using the macbooks, set up bootcamp so you can dual boot to OSX or Windows. This will mean you can use the Macbook for all development without having to bother swapping monitors, etc. Doing this will leave your other Windows laptops spare which you can use for the next suggestion....
Set up a central repository for your sourcecode. Use one of the servers you have, or re-purpose one of the other machines and install a decent source code repository system. CVS, Git, etc. There's plenty of resources about these. This will allow you to keep your code in one place so it won't matter which machine you are working on - you can always get the most recent code. Plus it will help you track your code changes. Oh, and don't forget having it all in one place will be much easier for backups (you do do backups, don't you....?)
Don't fall into the trap of upgrading hardware just because you have some money floating around. You may just need to use the hardware you have more wisely. You mention what you have is "far from professional". You don't need the latest, greatest hardware and software to do development. I've done iOS development on 4 year old Macbook Pro, used an 8 year old PC as a server for web and database and still use Windows XP every day.
Depending on how many of you there are, you may not have enough Macbooks. If this is the case, then perhaps you have some who are specialists in the server-side stuff (ie they don't do iOS development and so don't need the Macs).
Virtualisation - using VMWare or similar tools are an excellent way of getting more from what you have. For example, you could have a couple of test servers that aren't very heavily utilised. Using virtualisation, you could put both of these servers onto one machine. This will then free up the other box for something else. It also makes it very easy to backup (you are doing backups, aren't you...?) an entire server and recover it back to the exact state in the case of a hardware failure. You can also very easily create a server tailored for each client/project and switch between them quickly without having to maintain lots of other stuff (think if you had a web server configured for one project and you then work on another project that needs a different configuration and you change it, then you need to change it back, etc).
EDIT: Update in response to comments.
If using Bootcamp isn't an option, then consider running a Windows and/or Linux virtual machine inside OSX. Depending on the spec of your macbooks and as long as you don't need very low-level hardware access on Windows, then this would probably work as well and not need to switch in and out using BootCamp. Same goes for the Linux virtual machine. I'm a big fan of using Virtual Machines on development environments as it allows you to copy around and switch in and out servers without having to rely on physical hardware connections. And you can very easily return to a known state with the server configuration and data.
With regards the source control "in the cloud". I'm not a fan of this approach. It's my source code and I want to control it. I don't want to be reliant on some other company and I don't want to hope I've read some Terms and Conditions correctly and I'm not handing over my code to some other company to do what they want with it. Aside from that, what happens if your internet access goes down and you absolutely must get some coding done for a customer? If you are relying on another service, then you are risking problems. Yes, it has advantages for multi-site, they do the backups for you, etc. But it really isn't a problem unless you have lots of developers spread all across the world. And even then it isn't necessarily a problem. You could always do a backup of your code to some package file, encrypt it and then throw that up in the cloud for a backup storage (as well as burning it to disc, writing to another external hard drive and storing them off-site). But I certainly wouldn't want to rely on an external source control unless I was doing open source stuff.
There's sooooo much more to these subjects and there are many other subjects you will probably encounter along the way of building up your business.
One of the most important things about software development is to keep it organised and to get that organisation part done at the start. If you are just each keeping a copy of the code on local drives, then changing code and hoping that you haven't changed the same file as someone else, then this will just lead to pain. The source control aspect is key from the start.
Oh, and did I mention backups?
I would also consider the IDE you're using as part of the equation. For instance a good cross platform IDE (Like QT4+) and a centralised code repository on a server will go a long way towards mitigating your working problems. Eclipse, Netbeans and QT4+ are cross platform and will work with all 3 systems. Virtualisation as you mentioned is an option, but first I would decide on the IDE platforms to use before worrying about your dev infrastructure setup.
Bro, I'm not a pro, but you have two options:
Either multiboot your system by installing multiple OSes...(Obviously, you need a separate MACbook)
Or use Virtual Machines like VMWare etc.
Personally, I haven't heard much about libraries like Unity etc.
Go for dedicated systems & not just libraries.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
What are the problems with deploying an .EXE to a network drive and having users execute the .EXE over the network?
The advantage is that upgrades only need to be made to the one location. What are the disadvantages?
I would instead consider creating an MSI (http://en.wikipedia.org/wiki/Windows_Installer) file for your application and a Group Policy to facilitate distribution throughout your company (http://support.microsoft.com/kb/816102).
There are a number of freeware MSI tools. Good ones that come to mind are http://www.advancedinstaller.com/ and http://wix.codeplex.com/
The EXE is one thing, but you also need to consider any DLLs and other shared resources that may be associated with the app.
Some DLLs may be shipped with the EXE - you'd have to put those on the remote drive with the EXE, which would cause additional network traffic if it needed to use them.
Other DLLs may be part of Windows, but there could be versioning issues here if your workstations have different versions of windows or even different service packs or patches but they're all running a common version of the app.
And what about licensing? Does the app's license actually allow you to install it on a network drive - many software companies are very specific about this sort of thing, so you need to really be careful if you don't want to get caught out.
In short, it sounds like a good idea to get a quick win for your deployment management, but it probably causes far more issues than it solves.
If you really want to go down this path, you maybe should consider alternatives like remote desktop (eg Citrix or Terminal Server) or something like that - there are much better ways of achieving your goals than just sticking everything on a network drive.
One problem is file locking. In a Windows environment, if a user executes the application directly from a network share, the application's files are locked. This prevents the application from being updated with a newer version if someone has left the application open.
You can go around this by disabling the network share before updating the app and then again enabling it.
If you write your application using an Object Capability Security model, as defined in Mark S. Miller's Ph.D. thesis, Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control, then you will not have any security drawbacks.
On the other hand, the "disadvantage" is that you must now manage access control via the object graph. The application should only have access to whatever permissions you give it. As some have already mentioned, Windows has a basic protection policy which locks the application files and thus prevents anyone from modifying the EXE until the application instance(s) is closed.
Really, the key issue here is you have to ask yourself what authority the program and its component parts should have. If it requires local user permission, then you will either have to design around that or give the program permission.
Understanding the implications of this, and doing it well, is not an easy task.
For our program we decided against a shared exe. We thought it would be harder to support (IT needs to kill users to unlock files before updates, users wont know where the exe is on the network, share\network file permissions need to be modified by IT, etc) and that we should emulate the behavior of other programs when possible (client software is normally installed on the clients).
The main disadvantage would be the network drive being unavailable.
Then each language, which you didn't specify, the EXE is written in matters. As .NET has some security issues running from a network drive.
It depends on what the application does. My application would be a problematic over-the-network deployment because the configuration files it uses are all in the same folder as the EXE, or in a subfolder. If every user runs off of the network, they could potentially modify the configuration files and screw things up for everyone else.
Thankfully, my app is only going to be deployed on separate workstations. :)
They might not have all the files your app needs installed. If they don't, you'll need to create a setup. If they do and it works and everyone's drives are mapped correctly, you should be fine.
I run a vendor's app like this at work. They didn't design for it, but it works without an issue. I have all the shortucts pointing to the UNC path. This particular app doesn't use files in the exe directory, so file locking isn't an issue. Its also hooked up to SQL Server for the data, so the data store isn't an issue either. (Would be a major problem if the app used a local SQLite, Access, or some other file based DB.)
If your app is a .Net app, this WILL NOT work without some major modifications to each machine's security settings, which is probably bad idea anyway. If you're talking about a .Net app, you should use ClickOnce. I use it for a few apps at work, as well, and it's great, and easy to use.
The problem is there isn't a definitive answer to your question, just a bunch of "it depends" qualifications. The big issues, AFAIK, are using local files for data storage, be they text files or databases. It is awesome for updates, though, which is why the app mentioned above is run like this.
This is perfectly doable. Be sure to set the "Run from CD-ROM" (I think?) flag in the Visual Studio settings when compiling -- this prevents the image from being backed directly by the binary, so you can upgrade it while people are running it. I am not running Windows at the moment, so I can't check, but you may be able to set this flag for DLLs, too.
One problem with doing this is that if your program associates itself with files, when the network changes and computers are renamed everybody's PC starts to run like a dog. Explorer has a tendency to query these things at funny times.
Another more serious problem is that if somebody accidentally deploys a broken version, it's not just the early adopters who get stuffed!
For an easy life, personally I recommend XCOPY deployment...
For .NET applications, we have observed BadImageFormatException which we have come to believe is from network glitches (or computers loosing network connectivity at key moments, for example using WIFI) while reading the EXE or DLL files.
IMHO this is a really bad design decision. We have a third party application in our company which is designed exactly like this.
In order for the program to run properly it requires full sharing for that folder; In this case the worst part was that the program had the freaking DATABASE in the same shared folder (yeah, I was shocked too when I found out)!!! Didn't take too long till someone wiped every file that was not in use from that folder, including the database of course :)
I really recommend a client-server approach, even if you have to buy/build a smart installer with auto-update features to overcome deployment issues.
I'm just a hobbyist programmer more or less and have grown up coding-wise in the .NET ClickOnce world.
When one "installs" a program, what actually happens?!
Also: Some little apps/tools just run from the exe. Why do most programs need a fancy installation process? What are the advantages, disadvantages, pros & cons? Is installation usually necessary or more like standard practice?
Apologies for the extra questions. I'm just hoping for a plain-English more-or-less layman's explanation of the key factors.
You're really looking at a lot of legacy reasons all rolled into what has become standard practice in the Windows world.
First, some contrast, because it isn't always this way. An "application" in Mac OS X is simply a directory with a certain structure inside it, named with a .app extension. Installing an application is as simple as dragging it (just the app icon) to your Applications folder, and uninstalling involves dragging it to the trash. That's it, no fancy installer is (usually) necessary.
On Windows, application are typically built from independent components which need to be "registered". This involves the installer program writing some bits and pieces to the Windows registry, to tell Windows where to find the components. Yes, the application probably should know where to find them (since they're all installed in the same place), but years of legacy and different ways of hooking up components has got us where we are today.
Typically, an installation program on Windows:
copies files
registers components
sets security permissions (if appropriate)
adds icons to the Start menu and/or desktop
writes more stuff to the registry to tell Windows to add the program to "Add and Remove Programs"
The program tries to modify the computer in such a way that it works and all competing products fail. On Windows, this means:
Modifying arbitrary keys in the registry until it becomes slow and full of broken entries
Replacing DLLs with the single ancient version that your software can use
Spreading as many files in as many places as possible
Creating an uninstall script to maintain the illusion that the user can get rid of the software without a reinstall of the OS. In the unlikely case that the user tries to run this script, you can educate him/her not to ever do this again with questions like "The file .... might be used by other applications. Do you really want to delete it? Yes/No/Maybe/Any answer/All answers are correct"
Installing hooks in obscure places so your software runs when the computer boots. That may slow down the boot process but your software will start in an instant, so it's a small price to pay ... for you.
Doing obscure things which take a long time but no one can tell what you do (what does "Setup is preparing the install" do for 15 minutes?)
Checking whether there is enough disk space but use 32bit integers to make sure that it can't be installed on 1TB disks.
An important task is to fail with the installation and print the error: "Installation failed. This might be because there is an antivirus software installed. Please deactivate it and try again." This will make sure that users will start to distrust their anti-virus (especially when the install succeeds during the second run since the obscure bugs in the installer weren't triggered) and a lot of them will forget to enable the virus scanner again or even uninstall the damn thing.
Virus authors all over the world are people, too! Spam makes up for most of the traffic on the Internet which must mean that it's important and who wouldn't want to be part of the biggest community on earth? On top of that, you can make big money this way. All you need it a weak conscience and/or some criminal energy.
A very important part of your installer is to increase the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-7-9-23-64738-1349283462-3754093625-4491\IsYourWindowWideEnough\NotGivenUpYetHuh\GoAway\ImportantSystemInformation\Let See How You Can Handle Spaces\DamnIGottaStopSincePathsCanHaveOnl\ReinstalCtr
This important system counter will help to create the illusion of instability for the user until they feel a strong urge to reinstall the whole system. This will help the professional IT industry to sell support hours, sell new computers, more RAM, bigger hard disks, or new Windows versions (they must be better, right?).
Note: If you take this text seriously, seek professional help.
The reasons for using a "fancy" install process
to record the installation process
so it can be replayed (repair) or
undone (uninstall)
to perform actions beyons simple file copy (create registry keys, register components, perform other arbitrary actions
The "standard" option on most installs will be "all the bits usually wanted, in a standard location, like Program Files" install with no customisation, possibly without some expert-level features enabled.
An installer abstracts the process of deploying complex pieces of software infrastructure, which is usually contained within an archive, through a convenient, self-sufficient user interface.
This UI can be graphical or based on text which is output on a command-line such as the unix shell (e.g. bash). In case of graphical installers, most often a so called installation-bootstrapper is used, in the latter case, installation scripts which can be bash-scripts, Microsoft batch scripts, or other any scripting language which runs on a command line.
In the simplest case an application is simply an executable file, with the operating system knowing what to do with the file in order to run it. The application file may reside in a folder with subfolders and other auxiliary files, packed into one archive. In this case no installer may be needed.
For complex software, entire software platforms and tight integration with the underlying operating system infrastructure may be desirable, for instance to enforce the copyright of a software product.
Many installers on Windows provide an /e or /extract flag. e.g. setup.exe /e to allow extraction of the archive's contents without the installer running its installation script.
I recently needed to do just that.
Shifts in Mindset
Installers have almost become a norm for delivering professional software, no matter how simple the underlying software assets. With an increasing number of computer savvy users and the desire to migrate ones applications from one desktop to the next, portable software, often delivered in a simple archive, is becoming increasingly popular.
( I don't know how much time in total I have spent on installers, but it is definitely on the order of days. )
Tasks the installer may handle are:
unpacking (often using exotic, high compression archivers)
ensuring system hardware requirements
ensuring sufficient hard-disk space
ensuring software platform runtime requirements (e.g. 'redistributables')
checking for newer software updates
downloading the software from a remote repository
creating and/or updating program files and folders
create configuration files, registry entries or environment variables
install sofware drivers, mount or unmount devices
increase accessibility for everyday users, by explaining installation steps, creating links, shortcuts
promote the own sofware through bookmarks, etc...
create incentive for the user to actually startup the software, by presenting the keypoints of the software during the installation, slide by slide
create additional revenue, through software-bundling
configure kernel-modules and automatically running components (e.g. daemons, windows-services)
automatic patching of the sofware
setting folder, file and user permissions
creating UUIDs references to couple the software to an installation-instance and prevent portability
PS: If you can think of other points, let me know and I will incorporate them.
Wikipedia tells us that a typical installer creates or modifies the following:
Shared and non-shared program files
Folders/directories
Windows registry entries (Windows only)
Configuration file entries
Environment variables
Links or shortcuts
So if your program needs one or more of these modifications, you should create an installer which does that job.
Depends on the program you are installing. An "Installation" can range from simply copying the (relatively small) executable to a directory, to setting up shared libraries, doing patchlevel checks (I am designed to run on SP2 or higher - do I have SP2 or higher?) and changing the systems configuration, either for the current user or for all users. Most of them also register the Installation with a package manager so that you can easily uninstall at a later point.
I'm looking for a way to give out preview or demo versions of our software to our customers as easy as possible.
The software we are currently developing is a pretty big project. It consists of a client environment, an application server, various databases, web services host etc.
The project is developed incrementally and we want to ship the bits in intervals of one to two months. The first deliveries will not be used in production. They have the puropse of a demo to encourage the customers to give feedback.
We don't want to put burden on the customers to install and configure the system. All in all we are looking for a way to ease the deployment, installation and configuration pain.
What I thought of was to use a virtualizing technique to preinstall and preconfigure a virtual machine with all components that are neccessary. Our customers just have to mount the virtual image and run the application.
I would like to hear from folks who use this technique. I suppose there are some difficulties as well. Especially, what about licensing issues with the installed OS?
Perhaps it is possible to have the virtual machine expire after a certain period of time.
Any experiences out there?
Since you're looking at an entire application stack, you'll need to virtualize the entire server to provide your customers with a realistic demo experience. Thinstall is great for single apps, but not an entire stack....
Microsoft have licensing schemes for this type of situation, since it's only been used for demonstration purposes and not production use a TechNet subscription might just cover you. Give your local Microsoft licensing centre a call to discuss, unlike the offshore support teams they're really helpful and friendly.
For running the 'stack' with the least overhead for your clients, I suggest using VMware. The customers can download the free VMware player, load up the machines (or multiple machines) and get a feel for the system... Microsoft Virtual PC or Virtual Server is going to be a bit more intrusive and not quite the "plug n play" solution that you're looking for.
If you're only looking to ship the application, consider either thinstall or providing Citrix / Terminal services access - customers can remotely login to your own (test) machines and run what they need.
Personally if it's doable, a standalone system would be best - tell your customers install vmware player, then run this app... which launches the various parts of your application stack (maybe off of a DVD) and you've got a fully self contained demo for the marketing guys to pimp out :)
You should take a look at thinstall(It has been bought by vmware and is called thinapp now), its an application virtualizer.
It seems that you're trying to accomplish several competing goals:
"Give" the customer something.
Simplify and ease the customer experience.
Ensure the various components coexist and interact happily.
Accommodate licensing restrictions, both yours and the OS vendor's.
Allow incremental and piecewise upgrades.
Can you achieve all of these by hosting the back end (database, web server, etc.) and providing your customers with a CD (or download) that contains the client? This will give them the "download/upgrade experience" that goes along with client software, without dealing with the complexity of administering the back end.
For a near plug-and-play experience, you might consider placing your demo on a live linux or Windows CD. Note: you need a licensed copy of Windows for the latter.
Perhaps your "serious" customers might be able to request their own demo copies of the back end as well; they'd be more amenable to the additional work on their part.
As far as OS licenses, if your vendor(s) of choice aren't helpful, you might consider free or open-source alternatives such as FreeDOS or linux.
Depending on if you can fit all the needed services into a single OS instance or not...
Vmware Ace or whatever they're calling it nowadays will let you deliver single virtual machines under strict control, with forced updates, expiration and whatnot. But it sounds easier to just set up a demo environment and allow remote access to it.
The issue here I guess is getting several virtual machines to communicate under unknown circumstances - if one is not enough?
An idea then is to ship a physical server preconfigured with virtualisation and whatever amount of virtual servers needed to demonstrate the system.
Using trial versions of the operating system might be good enough for the licensing dilemma - atleast Windows Server is testable for 60 days, extendable to 240 when registering.
Thinstall is great for single apps, but not an entire stack....
I didn't try it yet, but with the new version of thinstall you are able to let different thinstalled application communicate.
But I guess you're right a vm-ware image would be easier