I get that Canopy "is its own ecosystem" as my friend told me (when he explained to me that I didn't have Python on my computer and that's why my command line instructions weren't doing anything). Could this be the reason that it takes longer to load than Eclipse, which (I guess) I downloaded separately from Java? Or is it some other reason?
Good question! Fast loading was not a design goal of Canopy. Most Canopy users leave it open for many hours at a time, so load time (while always nice, of course!) was not as high a priority as having it include a robust scientific Python distribution. The technical reason why it loads slowly (especially when the OS's cache is "cold") is that it opens and interprets many tens of Python modules, some quite complex. Unlike C, Python is usually not compiled, so the startup time of Python programs is typically slower, though when important, there are ways to counteract that.
Related
Some background:
I am a Computer Engineering major attending school right now, and I just completed a project that created a microprocessor with a super simple instruction set that ran on an FPGA. I chose to implement a simple file storage scheme, a VGA text only display output, and a PS/2 keyboard input. I wrote two main programs, a firmware that was in ROM in the processor and a kernel that provided a bunch of library type functions, and was capable of loading and executing files from the filesystem. This project was challenging and overall a lot of fun.
My Question:
I want to do some super low level programming on a modern computer, but I can't seem to find any resources or documentation that help me get started. To be clear, I want to find the proper documentation that would help me to write a program in C, x86, or x86-64 assembly that I could compile, and format into some form of bootable data. I know this is a daunting task, and typically not something a hobbyist would take on, but I know that it's possible (Terry Davis's TempleOS).
Are there any websites or books that I can read that will contain the specifics needed to make something like this?
Look out, you might just catch the bug. OS Development, though having a very small demographic, is still quite a thriving hobby. Once you start, you may not ever give it up.
Since your subject line states 64-bit and you use the term modern hardware, be advised that modern hardware no longer has the older style BIOS, where the developer wrote the boot process which included the video out, file system in, and other standard routines. Modern hardware now uses an EFI firmware which does all of the booting for you, including the reading from the file system(s). For modern hardware, OS development really starts with the OS Loader, the part that loads the OS, and this is done in a high level language such as C/C++. Very little if any assembly at all, in fact that is its point/purpose.
Don't get too discouraged, currently a lot of computers still allow the old style boot. However, the old style boot starts in 16-bit mode, moves to 32-bit mode, then if desired, moves to long mode (64-bit). There also are emulators that you can use so you don't have to have a separate system, just to test your development. I prefer Bochs myself, but I am a little bias since I wrote some of the code for it, namely most of the (original) USB emulation.
If you wish to dip your toes into this hobby, there are numerous places to start. I personally wrote a few books on the subject. They show you how to start from when the time the POST gives up control to your boot code, up to the point of a minimal Round Robin style task/thread switching OS, with all the necessary hardware and software basics to do so. There is a forum to OS Development, along with its wiki.
Again, a project like this is not for the faint at heart, though it is an enjoyable hobby most have found to be a very good learning experience.
I have recently found a Matlab program (TraitLab) that I would like to try and use, but I do not have a Matlab license. Given that I thought GNU Octave and Matlab are mostly compatible, I tried to run it in octave, but the GUI way to run the program depends on loading a .fig file (now I know that I can use load to get the struct content of that file, but that does not make it a figure I can pass around to various GUI functions), and the headless way to run it reports many parse errors about
nested functions not implemented in this context
Is there an obvious way to get a function written for Matlab that shows these two symptoms to run under Octave?
I have downloaded TraitLab and it is obvious to me that it is a program that will need heavy modification for you to run on octave, which you probably don't have time (or possibly expert knowledge) to implement.
In this case you really only have two other options.
1) Write to the developers and ask them if they could re-implement their code in such a way to make it Octave compatible. I wouldn't hold my breath since it's an academic library, but it's always worth pointing out that there is demand for Octave compatible code. Also given this is old code, it's still worth contacting the stats lab in oxford, since they may have moved to a different codebase that might also suit your needs (e.g. I know that a lot of stats people in oxford have made the switch to julia).
2) In the meantime, if you really want to run this but you don't have matlab, find a matlab installation (e.g. from a friend, or at your university) and create a standalone-version using Matlab's "Application Compiler".
OR
Consider buying matlab. It has a reputation for being prohibitively expensive, partly because University-wide licences do cost an arm and a leg. But for personal use (especially if you are a student?), with the right licence and only the necessary toolkits, it's actually not as prohibitively expensive as you might think.
Although I have implemented many projects in C, I am completely new to operating systems. I tried real time linux on Discovery board (STM32) and got the correct results for blinking LED but I didn't really understand the whole process since I just followed the steps and could not find whole description for each step on the internet.
I want to implement scheduling on real time linux. What is the best way to start? Any sites, books, tutorials available?
Complete RTLinux process description will be appreciated.
Thanks in adv.
The transition from "bare metal" to OS based programming is something that I experienced in reverse. I started out a complete software guy, totally into the OS side of things and over time I have moved to the opposite of that (even designing circuits in VHDL!). My advice would be to start simple. Linux is pretty complex, and everywhere you look there are many layers of things all working together to deliver the final product. If you are dead set on a real time linux extension, I'd be happy to suggest https://xenomai.org/ which is a real time extension for linux.
However, to more specifically address your question about implementing scheduling in Linux, you can, but it will be a large amount of work and can be very complicated. The OS uses a completely fair scheduling process ( http://en.wikipedia.org/wiki/Completely_Fair_Scheduler ) and whenever you spin up a thread, it simply gets added to the list to run. This can differ slightly if you implement your code in kernel space as a driver, rely on hardware interrupts, etc., but in general, this is how Linux works. Real time generally means that it has the ability to assign threads one of several different priorities and utilize thread preemption fully at any given time which are concepts that aren't really a part of vanilla Linux. It has some notion of this, but it has limitations that can cause problems when you are looking for real time behavior from Linux.
What may be helpful to you is an RTOS. If you are looking for a full on Real Time Operating System, check out FreeRTOS http://www.freertos.org/ . It has a large community and supports a lot of different devices out of the box with a large amount of example code. They even support your specific board with an example package, so you can give it a shot with nothing to lose! http://www.freertos.org/FreeRTOS-for-Cortex-M3-STM32-STM32F100-Discovery.html . It gives you access to many OS ish constructs like network APIs, memory management, and threading without the overhead and latency of a huge OS. With an RTOS, you create tasks and assign them priorities so you become the scheduler and are no longer at the mercy of the OS. You run the OS, not the OS runs you (if that makes sense). Plus, the constructs offered within an RTOS will feel much like bare metal code and thus will be much easier to follow, understand, and fully learn. It is a more simple world to learn the base building blocks of a full blown OS such as Linux or Windows. If this option sounds good, I would suggest looking through the supported devices on FreeRTOS website and picking one you would like to experiment with and then go for it. I would highly recommend this as a way to learn about scheduling and OS constructs in general as it is as simple as you can get and open source. Once you have the basics of an RTOS down, buying a book about Linux specifically wouldn't be a bad idea. Although there are many free resources on the web related to learning about Linux, they are commonly contradictory, and can be misleading. Pile on learning Linux specific knowledge along with OS in general, and it can feel overwhelming. Starting simpler will help keep you from getting burnt out and minimize the amount of time you spend feeling lost. Linux is definitely a learning process, but like with any learning process, start simple, keep your ultimate goal in mind, make a plan, and take small, manageable steps along that plan until you look up and find yourself exactly where you want to be. Then go tackle the next mountain!
The real-time Linux landscape is quite confusing. 99.99% of the information out there is just plain obsolete.
First, there are lots of "microkernels" that run Linux as one task. (Such as the defunct RTLinux). The problem is that you must write your real-time task to a different API, and can't depend on anything in Linux, because Linux will be frozen in the background while your task runs. So unless your task is dead-simple ("stop the motors when I press this button"), this approach will cause more pain than gain.
Next, there is the realtime Linux patch set. This hasn't been doing so well. because of the next item:
Lastly, the current Linux kernel has gotten rid of the problems that caused people to need realtime in the past. You can even turn off Linux on one of your processors to have full control of the CPU. See also this paper.
To answer your question: I see two different paths you could take:
1) Start with a normal 3.xx Linux kernel and explore the various APIs and realtime techniques (i.e. realtime priorities, memory pinning, etc.) This can get you "close enough" for 99% of what people want "realtime" for. If it's good enough for high frequency trading, it's probably good enough for you.
2) If you have a hard realtime requirement and you are worried that Linux won't cut it, then (as Nick mentioned above), just go buy a processor and write your realtime code with no OS. By splitting up your "realtime" and "non-realtime" code onto different CPUs, you will make the whole system simpler and much more robust.
If you want to learn real-time operating systems then I suggest that you get an FPGA, for example the Altera DE2, and experiment with your own operating system and ucos. You can read a good text about embedded RTOS here.
You could also get a Linux Raspberry and write your own operating system for that.
Does anyone know of a scripting language that's included with most platforms (say Mac/Windows/Linux)? I haven't been able to find one. So far javascript in web browsers or compiled java are about it. Jython comes close.
My goal is to be able to download a file from the web or portable storage and just run it, without having to install something first, or have special user permissions, or edit it, or rename it, or give it executable privilages. It would give you access to generally accepted metaphors in computing: input, output, persistent storage, time, spawning tasks, sockets, fixed and floating point math, unicode, etc. Ideally it would abstract away minutia like line endings, endianness, and yielding for other processes.
I don't want to get into why having a universal language/virtual machine is important, or at the very least, useful. I feel that we are missing a middleware above the operating system level, something like POSIX but less esoteric, and without it, we all are forced to spend a disproportionate amount of time reinventing the wheel or writing special cases. For me, availability and a complete feature set are more important than speed (which could come later).
Thanks in advance for any insights you can provide,
Zack Morris
You will be able to run carefully written sh scripts on almost all unix systems.
If you want to add Microsoft systems, then it is more difficult, but still possible to provide a single script file, that will "autodetect" the interpreter it's running on and select between a sh part and a command.com or whatever they have on Microsoft systems.
Once you can run a script on a known system you can further download or unpack and install automatically whatever software you need.
I have a stock mac-mini server (2.53GHz, 4GB, 5400RPM drive) which I use for iphone development. I am working on a smallish app. The compilation seems to be taking longer and longer (now about 20min). The app is about 20 small files totaling 4000 lines. It links to C++/Boost.
What suggestions do you have to speed up the compilation process? Cheaper will be better.
Sounds like the system is swapping, which is a common problem when compiling C++ that is STL heavy. GCC can be a complete memory pig in such a situation and two GCCs at once can easily consume all memory.
Try:
defaults write com.apple.Xcode PBXNumberOfParallelBuildSubtasks 1
And see if that helps.
If it doesn't, then the other answers may help.
As others have noted, that's way in excess of what you should be expecting, so it suggests a problem somewhere. Try toggling distributed builds -- depending on your setup it may improve things or slow them down (for a while we had a situation where one machine was servicing everyone else's builds but not its own). If you have a 1Gb network then you may get a bit of extra performance from distributed builds (especially if you have a couple of spare macminis). Also enable precompiled headers, and put both boost and cocoa headers in it.
If things are still slow then it's worth running activity monitor to ensure that you don't have some other process interfering, or even Instruments to get some more detailed information. You could find that you're trying to compile files on a network drive for instance, which will certainly slow things down. Also inside Xcode try running the option to preprocess a file and see what's in the file contents (available via right-click) to check that you're not compiling in loads of extra code.
Also with boost, try slimming it down to find only the bits you need. The bcp tool that comes in the distribution will copy only the packages you ask it to, plus all the dependencies, so you're not building stuff you won't use.
Include all your boost headers in the precompiled header for your project.
I concur with Eiko. We're compiling projects that are vastly larger (including boost libraries) in much shorter time.
I'd start by checking logs. You can also view the raw compiler output in the build window (Shift+CMD+B or in the Build menu->Build Results). Xcode just makes straight gcc or llvm calls, so this may give you a better idea of what's actually happening during the build process.
There is something messed up. We are compiling bigger projects in fractions of the time.