I have a Spark project which uses HDFS. I need to edit some things in the source code of Spark, which is possible since it is open source.
However, I'm wondering on how to get all of this together and working. I downloaded the source-jar for Spark and attached it to my IntelliJ project, which by the way builds with Maven. This has given me the possibility to go from my code and for example use Ctrl+B to go and see the source code of SparkContextor any other code in the Spark library.
Anyhow, this code is locked and I cannot modify it. I have not found a way to unlock it. So after some searching on the internet it seems like the solution would be to find a way to build the whole Spark library as part of my project instead of using it as an external library.
How would I go by doing this? I am lost as to how to add it so the source code is part of the project, which package would it be a part of and so on? I have no idea where to start or how to do it at all.
You are probably better off checking out the latest source code for Apache Spark, create a feature branch, put your changes there and then probably submit a merge request, so that it is officially available as part of the Apache packages.
If you do not want to do that, you could of course do it all by yourself, with just building the Spark package by yourself, having it somewhere in your company's internal repository so that you can pull it from there from your different environments.
Related
I am using this code: https://github.com/pradeepramakrishna/Lightning-Experience/tree/master/aura-components to compile on my org.
I have got one components as eventLib.lib and interactive.js in eventLib folder. I have put this folder in c:MyDevOrg\force-app\main\default\aura folder in the local machine and tried to deploy in the org using VSCode but it didn't work.
I tried creating folder in the Files tab and added the interactive.js in the folder but that is also not seeming to work with the Aura components.
How can I deploy this lib into the org as it is being used in some Aura components.
We don't know what you're trying to achieve. You asked a very technical question to which the answer is "you're probably doing it wrong". Instead try to write what business functionality you are trying to achieve and you might get better answers.
You shouldn't have to import ui:eventLib. It's supposed to be part of core Salesforce's Aura components framework. But.
You've referenced a repo that wasn't updated in a while, no promises this still compiles / is best way to do X.
This repo seems to rely on open source Aura framework which is well... dead in the water.
You might be able to reuse something from this repo in your app using SF's built-in tags - but whole ui: library has been deprecated. Announced in Winter'20 = almost 2 years ago, finally dead in May 2021.
I don't think eventLib was ever exposed, might be something needed just in open source version which complicates the matter more. The answer would be to not work with the really "decompiled" tags but with their higher abstracted versions like ui:inputDate.
So, back to my question. What exactly are you trying to achieve.
You want to build something on pure SF platform (use pure Aura/LWC).
Want to have an app written in Angular, React etc, pure JS, connecting to SF data via API? (build it, upload as static resourcethen import using lightning:container)
want to prettify an existing Java/PHP/.NET app, make it look more Lightning-ish and embed it as iframe? (look into https://www.lightningdesignsystem.com/ and connected apps + "canvas")
want to expose piece of SF as reusable element that can be embedded in another website but could even be an Outlook plugin? (search for "lightning out")
want to look at modern equivalent of that old open source Aura repo and decide what to do next? Check https://lwc.dev/ out.
I'm working on a scala sbt project, and I am at a point where I want to assemble the whole thing to share a .jar-file with others, so that they can use it on their side. For my local testing, I'm doing so using the sbt-assembly plugin, that works nicely.
When sharing, I would though prefer to only share the parts that are important for the other party (the project has huge components that irrelevant at the current point, and I'd prefer not to share these for various reasons). Concretely, they will be executing one particular main file, so it would be enough to pack everything that this file depends on.
Is there a way to accomplish this? I'd also be interested in doing this on the code level (i.e., create a copy of the project that only contains dependencies of that main file), but also while assembling or even modifying the jar file after assembly is okay. I did not find the tools to achieve any of these.
As I said, I'm dealing with a scala sbt project, and I'm working with IntelliJ IDEA; I'd also be happy with an IDE tool that does the job.
When writing a contract for an API i found myself repeating the same things over and over. For example regex validations for complex json object need to be copy pasted.
Thats tedious and not very DRY.
I'm using Kotlin DSL and Maven
I tried to extract the common parts in another file to reuse it. (Kotlin extension functions ftw.)
After trying multiple things to reuse that file I gave up.
The common parts should be as close to the actual contracts as possible. I don't want to export them in another project, and build them seperat, as they are an vital part of the contracts.
I tried it the following ways:
just put the file in the same directory and importing the functions, hoping it would be resolved as it would in java - did not work at all (my expectations were low, was worth a shot)
putting it in another maven module, and add that as dependency to the spring-cloud-contract-maven-plugin. that worked, as long as the dependant module was built and installed in the local maven repo. if no built version was available maven could not resolve it.
experimenting with kotlin script #file:Import() and #file:DependOn to tackle my issue, no luck.
Is there another way, that I missed? Is there a prefered way doing this?
This must be a common issue, right?
We use gradle as our build tool and use the idea plugin to be able to generate the project/module files. The process for a new developer on the project would look like this:
pull from source control.
run 'gradle idea'.
open idea and be able to develop without any further setup.
This all works nicely, but generally only gets exercised when a new developer joins or someone gets a new machine. I would really like to automate the testing of this more frequently in the same way we automate our unit/integration tests as part of our continuous integration process.
Does anyone know if this is possible and if there is any libraries for doing this kind of thing?
You can also substitue idea for eclipse as we have a similar process for those that prefer using eclipse.
The second step (with or without step one) is easy to smoke test (just execute the task as part of a CI build), the third one less so. However, if you are following best practices and regenerate IDEA files rather than committing them to source control, developers will likely perform both steps more or less regularly (e.g. every time a dependency changes).
As Peter noted, the real challenge is step #3. The first 2 ones are solved by your SCM plugin and gradle task. You could try automating the last task by doing something like this
identify the proper command line option, on your platform, that opens a specified intellij project from the command line
find a simple good enough scenario that could validate that the generated project is working as it should. E.g. make a clean then build. Make sure you can reproduce these steps using keyboard shortcuts only. Validation could be made by validating either produced artifacts or test result reports, etc
use an external library, like Robot, to program the starting of intellij and the running of your keyboards. Here's a simple example with Robot. Use a dynamic language with inbuilt console instead of pure Java for that, it will speed your scripting a lot...
Another idea would be to include a daemon plugin in intellij to pass back the commands from external CLI. Otherwise take contact with the intellij team, they may have something to ease your work here.
Notes:
beware of false negatives: any failure could be caused by external issues, like project instability. Try to make sure you only build from a validated working project...
beware of false positives: any assumption / unchecked result code could hide issues. Make sure you clean properly the workspace, installation, to have a repeatable state and standard scenario matching first use.
Final thoughts: while interesting from a theoretical angle, this automation exercise may not bring all the required results, i.e. the validation of the platform. Still it's an interesting learning experience and could serve as a material for a nice short talk, especially if you find out interesting stuff. Make it a beer challenger with your team when you have a few idle hours to try to see who can implement the fastest a working solution ;) Good luck!
Situation:
I need several swf/exe output files compiled in FlashDevelop from several projects. More than 60% of ActionScript 3.0 source is common for all project, rest are project-specific. How can I organize that in FlashDevelop? I want to have "one-click-to-build all" setting without duplicating common codebase (so when I need to fix something I do not need to copy-paste solution into several files).
All sources are under develeopment and will change very often.
A straightforward solution is to make an external classpath, for instance:
c:\dev\shared_src\
c:\dev\project1\
c:\dev\project2\
Then configure each project:
Project Properties > Classpath
Add Classpath > select '../shared_src'
PS: of course you should keep everything under source control.
Using svn:externals you could structure your repository in such a way that the commom parts are stored just once in the source control system, so changes made can be synchronised with just a single commit and update cycle.
For example, imagine that you have ^/ProjectA and ^/ProjectB, each of with require ^/Common as a sub directory.
Using svn:externals, pull ^/Common into both projects.
The exact nature of doing this will depend on the version of svn you use, and any client you use (such as TortoiseSvn). Refer to the relevant edition of the svn book for specifics.
The ease of implementing this will depend quite a lot on how separate the common code currently is in your application; and pulling in directories as directories is much more practical than trying to pull them into an existing directory; and unfortunately wildcards for filepaths are not supported.
However, based on your description of your aim; this is the most straight-forward solution I can imagine.
Hope this helps.