Seth's comment below listing a URL in the maven repo, seems to be both correct, and general. Following the logic, I also found the spark docs at:
http://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.2.1/spark-core_2.10-2.2.1-javadoc.jar
Which means that in effect my question is answered, but since Seth didn't give an "answer" I can't mark it as such. Not sure what to do about that!
----------------- original question below -----------------
I fear I'm being unobservant or obtuse, but I have been unable to find where I can download the API docs for Scala, and also for Spark. I know where they are online, and I suppose I might try a recursive "wget", but I'm leery of that since I'm not sure I could get it to produce anything that works locally anyway. Perhaps I need to build them locally, but that too seems like a lot of effort since I can't quite believe these resource don't exist in a ready-made form.
In case this seems like a stupid request, I spend half my life on the road, on aeroplanes with expensive, low quality, (and theoretically unnecessary!) internet connections, or in the middle of nowhere, so I'm in the habit of installing all my tools and resources, particularly including API docs locally. That was easy with Java, but not so obvious with these two technologies.
Can anyone tell me where to find a downloadable package for each, or if I do in fact have to use one of the techniques I mentioned in my opening paragraph?
Seth, who made the first comment under my original question nailed this. I'm adding this essentially so that the question shows as answered.
In summary:
Both documentation sets are downloadable from the maven repositories:
http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.4/scala-library-2.12.4-javadoc.jar
and
http://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.2.1/spark-core_2.10-2.2.1-javadoc.jar
It appears that this is a general observation that's likely applicable to many if not most packages that maven has to offer.
Related
I'm using ScalaJS with Play. Many of the models I'd like to use on both JS and JVM platforms involve dates and times. Given the lack of a cross-platform date/time library, how are people approaching this?
Things I know about:
scalajs-java-time project (https://github.com/scala-js/scala-js-java-time) to port JDK8's java.time api to Scala.js. Unfortunately, it's far from complete and judging by the commit logs, seems to have stalled.
https://github.com/mdedetrich/soda-time is a port of JodaTime to Scala/Scala.js. But it's not ready for production use.
An old post at https://groups.google.com/forum/#!topic/scala-js/6JoJ7x-VxLA suggests storing milliseconds in shared code and then doing implicit conversions on each platform to either js.Date or JodaTime. But we really need a common interface, which this doesn't give.
Li Haoyi's excellent "Hands-on Scala.js" has a simple cross-platform library (http://www.lihaoyi.com/hands-on-scala-js/#ASimpleCross-BuiltLibrary) that could, in theory, be extended to come up with an API in /shared that delegates to JodaTime on the jvm and Momento on js -- but that sounds like a lot of work.
(added later) https://github.com/soc/scala-java-time is based on an implementation of java-time that was contributed to OpenJDK. The README claims that most stuff is working. Right now, this looks like the most promising approach for my needs.
Any advice from those who have gone before me? Right now the fourth options seems like my best bet (with the API limited to stuff I actually use). I'm hoping for something better.
I was in the same boat as you, and the best solution I came up with was cquiroz's scala-java-time library. From reading the comments to your question above, it appears you landed at the same place eventually!
I came here from a google search, and given how much better this solution is than the alternatives you mentions above, let's consider marking this question as resolved for future visitors.
I want to learn Lift. Unfortunately, all documentation which I tried either obsolete, unreadable, incorrect or combination of the above. I tried the following:
Simlply Lift. Some things from the book I tried lead to errors.
Exploring Lift. The structure of book is very bad. It's hard to read and try out code in the wild at the same time.
Lift in Action. The same as the previous but you need to pay for it.
P.S. I've seen similar questions. Most of them were asked a long time ago. Did the situation improve from the time of that writings?
P.P.S. Are there any other type safe scala web frameworks (Don't offer Play 2.0. It's not typesafe. I don't see any reason to create it in Scala).
It is unfortunately true that the state of Lift documentation is uneven at best and there are huge gaping holes.
However, the Lift community is just full of awesomely helpful people.
My recommendation is not to play around, but rather to try and implement something. If you get stuck, ask specific, direct questions about what you're trying to do, how you're doing it and why it isn't working.
So far, though I would wish for better documentation, I've been able to get every answer that I needed either through Google or on the Lift mailing list - though I expect I might ask more questions here in the future.
The Lift documentation is not its strong point. The philosophy is more "try and ask if you have any problem". Here are a few tips:
Assembla
One ressource that is really useful is http://www.assembla.com/wiki/show/liftweb, there are a lots of examples so you can progressively learn how it works.
Mailing List
Otherwise you can always use the mailing list if you have specific questions even if in my opinion it is really hard to explore it fast in order to solve a problem which was already encountered. http://groups.google.com/group/liftweb
Stack Overflow
Finally, a small community is present on Stackoverflow so feel free to ask in here. This is a good way of looking for answers and creating documentation in the same time.
Source code
Don't hesitate to explore the source code and the scaladoc if you have specific questions/doubts about the behavior of a function, they are often short and even sometimes commented! http://scala-tools.org/mvnsites/liftweb-2.4-M4/#package
Have a look at the Lift Cookbook: http://cookbook.liftweb.net/
"Simlply Lift. Some things from the book I tried lead to errors."
What exact type of errors did you have? Have you tried to follow it with "Simply Lift" examples that you can download from GitHub
https://github.com/dpp/simply_lift?
Only errors I had were related to my lack of experience with SBT, but that's another story.
I have started with Lift mostly from that source (Simply Lift + examples) and in combination with its great community and Google (ChrisJamesC has listed the main links really nice) it was quite okay for me.
I would suggest you to work out all examples given in the "Simply Lift" tutorial or at least work them out unless you feel comfortable enough to jump right "in media res" and try something by yourself. That was the best way of learning Lift for me.
Also, whenever you got stuck somewhere and can't find solution on the web, your questions would be welcome and answered on the Lift Google Group (https://groups.google.com/forum/?fromgroups=#!forum/liftweb). David Pollak is very often right there to answer your questions directly so I have only words of praise for this framework's community and Lift's
creator.
P.S. Lift's documentation could be better organized, some stuff could be better explained for sure, but IMHO it was just too small a price I had to pay to enjoy such beautiful framework. Learning curve is steeper than with Play, especially in the beginning, but after I "survived" the very first week it was almost impossible for me to give up of all of its advantages and original concepts (Lift's "Seven Things") and switch to another framework.
Despite having read "Programming in Scala" several times, I still often finds important Scala constructs that were not explained in the book, like
#uncheckedVariance
#specialized
and other strange constructs like
new { ... } // No class name!
and so on.
I find this rather frustrating, considering that the book was written by the Scala "inventor" himself, and others.
I tried to read the language specification, but it's made for academics, rather than practicing programmers. It made my head spin.
Is there a website for "Everything "Programming in Scala" Didn't Tell You" ?
There was the daily-scala Blog, but it died over a year ago.
Currently, we're working on a central documentation site for scala-lang.org. We're hoping that this solves a lot of the documentation issues that new users face. More details on this effort can be found at http://heather.miller.am/blog/2011/07/improving-scala-documentation/, but in summary...
Believe it or not, there are a lot of documents that the Scala team has produced but which simply aren't in HTML or are otherwise difficult to find. Such as Martin's new Collections API, his document on Arrays, or Adriaan's on Type Constructor Inference.
One goal of such a site is to collect all of this documentation in one place, in a searchable, organized, and easy-to-navigate format.
Another goal is to collect excellent community documentation out there, and to put it in the same place as well. For that, we are actively looking for quality (article/overview-like) material with maintainers. Examples include the Scala Style Guide, and Daniel Spiewak's Scala for Java Refugees.
Yet another goal is to make it easy for contributors to participate- so the site is built from RST source, which will live in a documentation-only github repo at https://github.com/scala/scala-docs.
So, in short, something better is on it's way, and contributors are very welcome to participate.
EDIT: http://docs.scala-lang.org is now live.
Several documents considered to be rather detailed or even obscure are already available. This includes all "Scala Improvement Proposals" (the proposals produced when new language features are suggested, and which are usually very detailed, and written by the implementers themselves). Also available is the entire glossary from Programming in Scala, Scala cheatsheets, amongst many other documents. The bottom-line of the site is to be community-focused and contribution-friendly-- so, free, and totally open. Suggested topics to cover are also welcome.
Take a look at scalaz and typelevel librairies (shapeless, spire, etc.), they rely on many advanced features of Scala.
*scalaz was for a time part of typelevel, but it is no more the case.
Josh Sureth's book goes a little beyond the usual. It's not as far as I'd like but I'm not his core audience - still, there's a lot of good stuff in there.
http://www.manning.com/suereth
Scala IRC: irc://irc.freenode.net/scala
Scala forum: http://scala-forum.org/
Blogs: Just look at http://planetscala.com/
Programming Scala (Wampler, Payne): http://ofps.oreilly.com/titles/9780596155957/
Programming in Scala (Odersky, Venners, Spoon) - good but Scala 2.8: http://www.artima.com/pins1ed/
The new documentation page is online:
http://docs.scala-lang.org/
I've kept a library of advanced Scala resources, primarily talks and blog posts. It's updated pretty regularly as I find new, interesting content.
Happy to add new links to it if anyone has recommendations.
Try to read SBT Source: https://github.com/harrah/xsbt/wiki
Its a good exercise. Also check out the book 'scala in depth' : http://www.manning.com/suereth/ by
Joshua D. Suereth
I believe there are a lot of good answer here. But as a sharing of experience. I have been coding Scala for 2 year (not my full time job), and been progressively better at it. My project is 97% Scala, and I have been able to do most of it with:
Programming Scala
The scala-user list
Stackoverflow
This cover most of the need for the "user" side of Scala, meaning all you need to create working application. However if you want to write some more complex code, or create powerful typed libraries you definitely need more.
If you want to go beyond the basics and are prepared to delve deeply into type system, and libraries, then the alternatives I use:
Use the community, scala enthusiast are really nice. I have worked with folks form Specs, Scalaz and Lift.
IRC is really good and some of the core contributors to some of the big library frequently show up.
Jump to source code, but don't try to understand everything. Scala type system can be daunting, however you normally don't need to understand 100% of it to use it.
If you really need to get into the nitty gritty details, hit the language specs, development list, and get to know the key people.
However you can really be very effective in Scala without needing to understand every single bit of the language.
I'm planning on starting a new project, and am evaluating various web frameworks. There is one that I'm seriously considering, but I worry about its lasting power.
When choosing a web framework, what should I look for when deciding what to go with?
Here's what I have noticed with the framework I'm looking at:
Small community. There are only a few messages on the users list each day
No news on the "news" page since the previous release, over 6 months ago
No svn commits in the last 30 days
Good documentation, but wiki not updated since previous release
Most recent release still not in a maven repository
It is not the officially sanctioned Java EE framework, but I've seen several people mention it as a good solution in answers to various questions on Stack Overflow.
I'm not going to say which framework I'm looking at, because I don't want this to get into a framework war. I want to know what other aspects of the project I should look at in my evaluation of risk. This should apply to other areas besides just Java EE web, like ORM, etc.
I'll say that so-called "dead" projects are not that great a danger as long as the project itself is solid and you like it. The thing is that if the library or framework already does everything you can think you want, then it's not such a big deal. If you get a stable project up and running then you should be done thinking about the framework (done!) and focus only on your webapp. You shouldn't be required to update the framework itself with the latest release every month.
Personally, I think the most important point is that you find one that is intuitive to your project. What makes the most sense? MVC? Should each element in the URL be a separate object? How would interactivity (AJAX) work? It makes no sense to pick something just because it's an "industry standard" or because it's used by a lot of big-name sites. Maybe they chose it for needs entirely different from yours. Read the tutorials for each framework and be critical. If it doesn't gel with your way of thinking, or you have seen it done more elegantly, then move on. What you are considering here is the design and good design is tantamount for staying flexible and scalable. There's hundreds of web frameworks out there, old and new, in every language. You're bound to find half a dozen that works just the way you want to think in your project.
Points I consider mandatory:
Extensible through plug-ins: check if there's already plug-ins for various middleware tasks such as memcache, gzip, OpenID, AJAX goodness, etc.
Simplicity and modularity: the more complex, the steeper the learning curve and the less you can trust its stability; the more "locked" to specific technologies, the higher the chances that you'll end up with a chain around your ankle.
Database agnostic: can you use sqlite3 for development and then switch to your production DB by changing a single line of code or configuration?
Platform agnostic: can you run it on Apache, lighttpd, etc.? Could you port it to run in a cloud?
Template agnostic: can you switch out the template system? Let's say you hire dedicated designers and they really want to go with something else.
Documentation: I am not that strict if it's open-source, but there would need to be enough official documentation to enable me to fully understand how to write my own plug-ins, for example. Also look to see if there's source code of working sites using the same framework.
License and source code: do you have access to the source code and are you allowed to modify it? Consider if you can use it commercially! (Even if you have no current plans to do that currently.)
All in all: flexibility. If I am satisfied with all four points, I'm pretty much done. Notice how I didn't have anything about "deadness" in there? If the core design is good and there's easily installable plug-ins for doing every web-dev 3.0-beta buzzword thing you want to do, then I don't care if the last SVN commit was in 2006.
Here are the things I look for in a framework before I decide to use it for a production environment project:
Plenty of well laid out and written documentation. Bad documentation just means I'm wasting time trying to find how everything works. This is OK if I am playing around with some cool new micro framework or something else, but not when it's for a client.
A decently sized community so that you can ask questions, etc. A fun and active IRC channel is a big plus.
Constant iteration of the product. Are bugs being closed or opened on a daily/weekly basis? Probably a good sign.
I can go through the code of the framework and understand what's going on. Good framework code means that the projects longterm life has a better chance of success.
I enjoy working with it. If I play with it for a few hours and it's the worst time of my life, I sure as hell won't be using it for a client.
I can go on, but those are some primary ones off the top of my head.
Besides looking at the framework, you also need to consider a lot of things about yourself (and any other team members) when evaluating the risks:
If the framework is a new, immature, "bleeding-edge" framework, are you going to be willing and able to debug it and fix or work around whatever problems you encounter?
If there is a small community, you'll have to do a lot of this debugging and diagnosis yourself. Will you have time to do that and still meet whatever deadlines you may have?
Have you looked at the framework yourself to determine how good it is, or are you willing to rely on what others say about it? Why do you trust their judgment?
Why do you want to use this rather than the "officially sanctioned Java EE framework"? Is it a pragmatic reason, or just a desire to try something new?
If problems with the framework cause you to miss deadlines or deliver a poor product, how will you talk about it with your boss or customer?
All the signs you've cited could be bad news for your framework choice.
Another thing that I look for are books available at Amazon and such. If there's good documentation available, it means that authors believe it has traction and you'll be able to find users that know it.
The only saving grace I can think of is relative maturity. If the framework or open source component is mature, there's a chance that it does the job as written and doesn't require further extension.
There should still be a bug tracker with some evidence of activity, because no software is without bugs (except for mine). But it need not be a gusher of requests in that case.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
In my team we've got a great source control system and we have great specs. The problem I'd like to solve is how to keep the specs up-to-date with the code. Over time the specs tend to age and become out of date
The folks making the specs tend to dislike source control and the programmers tend to dislike sharepoint.
I'd love to hear what solutions others use? is there a happy middle somewhere?
Nope. There's no happy middle. They have different audiences and different purposes.
Here's what I've learned as an architect and spec writer: Specifications have little long-term value. Get over it.
The specs, while nice to get programming started, lose their value over time no matter what you do. The audience for the specification is a programmer who doesn't have much insight. Those programmers morph into deeply knowledgeable programmers who no longer need the specs.
Parts of the specification -- overviews in particular -- may have some long-term value.
If the rest of the spec had value, the programmers would keep them up to date.
What works well is to use comments embedded in the code and a tool to extract those comments and produce the current live documentation. Java does this with javadoc. Python does this with epydoc or Sphinx. C (and C++) use Doxygen. There are a lot of choices: http://en.wikipedia.org/wiki/Comparison_of_documentation_generators
The overviews should be taken out of the original specs and placed into the code.
A final document should be extracted. This document can replace the specifications by using the spec overviews and the code details.
When major overhauls are required, there will be new specifications. There may be a need to revisions to existing specifications. The jumping-off point is the auto-generated specification documents. The spec. authors can start with those and add/change/delete to their heart's content.
I think a non-Sharepoint wiki is good for keeping documentation up to date. Most non-technical people can understand how to use a wiki, and most programmers will be more than happy to use a good wiki. The wiki and documentation control systems in Sharepoint are clunky and frustrating to use, in my opinion.
Mediawiki is a good choice.
I really like wikis because they are by far the lowest pain to adopt and keep up. They give you automatic version control, and are usually very intuitive for everyone to use. A lot of companies will want to use Word, Excel, or other types of docs for this, but getting everything online and accessible from a common interface is key.
As much as possible the spec should be executable, via rspec, or doctest and similar frameworks. The spec of the code should be documented with unit tests and code that has well named methods and variables.
Then the spec documentation (preferably in a wiki) should give you the higher level overview of things - and that won't change much or get out of sync quickly.
Such an approach will certainly keep the spec and the code in sync and the tests will fail when they get out of sync.
That being said, on many projects the above is kind of pie-in-the-sky. In that case, S. Lott is right, get over it. They don't stay in sync. Look to the spec as the roadmap the developers were given, not a document of what they did.
If having a current spec is very important, then there should be specific time on the project allocated to write (or re-write) the spec after the code is written. Then it will be accurate (Until the code changes).
An alternative to all of this is to keep the spec and the code under source control and have check-ins reviewed to ensure that the spec changed along with the code. It will slow down the development process, but if it is really that important ...
One technique used to keep the documentation in sync with the code is literate programming. This keeps the code and the documentation in the same file and uses a preprocessor to generate the compilable code from the documentation. As far as I know this is one of the techniques Donald Knuth uses - and he's happy to pay people money if they find bugs in his code.
I don't know of any particularly good solution for precisely what you're describing; generally, the only solutions that I've seen that really keep this sort of stuff in sync are tools that generate documentation from the source code (doxygen, Javadoc).