I'd like to download a website to whiten it into a private network.
I know it's being done a lot with StackOverflow itself, I just don't know how to do it myself.
The specific site I want to download is CPP-QUIZ with all the questions and explanations.
I've tried doing it with HTTrack, but it seems to download just a couple of questions and then it fails.
What is usually done to do something like this?
You can use a tool like wget or curl. If the site had an index then you could use wget's recursive option. It seems like it does not (the homepage just seems to choose a question at random for that specific site).
In that case you can just start generating commands:
wget http://cppquiz.org/quiz/question/1
wget http://cppquiz.org/quiz/question/2
wget http://cppquiz.org/quiz/question/3
And so on. After running those commands you would have the files downloaded to the directory you ran the commands from.
I was also wondering the same thing & stumbled upon this Github project this week called Diskernet
it saves resources to your file system as your browsing using your web browser (when launched with the diskernet binary or CLI tool)
It's still an early project, but I found I already started making use of it this week & it worked great!
https://github.com/i5ik/Diskernet
Highlights:
is able to download Single Page Web applications (doesn't use wget)
once you download a site, it works offline
There are alternative tools like SiteSucker, but I found Diskernet to work even for websites that require authentication
Related
Is there any complete guid for install postfix admin rather than official install guid?
I tried to install postfix admin several times. But I couldn't. I tried with official installation text file and several online tutorials.
Installing PostfixAdmin is not very hard, in-fact it's very simple.
PostfixAdmin is basically just a Apache+PHP+MySQL setup.
The bigger problem I think users are experiencing is how to make Postfix actually use the database backend and to "understand" the data it gets from it - What they refer to as the integration part in their INSTALL.txt for PostfixAdmin.
Take a look at the Postfix integration documentation at their GitHub page.
https://github.com/postfixadmin/postfixadmin/blob/master/DOCUMENTS/POSTFIX_CONF.txt
There are no easy way to just copy-pasta aka spray-and-pray install a complete SMTP/Mail system. Also I would not recommend it as you would have a very hard time to debug problems in your setup later on - And trust me SMTP servers always have problems :)
But basically once you have installed PostfixAdmin and that part works (i.e you can log onto the page, navigate around postfixadmin and add domains and so on) then you move on to adding all the .cf files it needs in the postfix etc directory.
Again, use that file above as reference. Or even search for multiple guides on Google. You will see that more or less all of them are doing the same steps.
Once your postfix is setup to talk "mysql" you can start to debug the mail.log files for problems that will (again trust me they will) come up. That will typically be beginner mistakes like the .cf files are not looking up the right data in the tables or using wrong parameters for email vs domain or that postfix don't have mysql support enabled and so forth.
In previous versions of AEM, certainly in CQ 5.6 and AEM 6.0, it was possible to tail the error logs over HTTP, without connecting to the server over SSH.
For example, I could get the last 1000 lines from the error log of my AEM author instance by calling:
http://localhost:4502/bin/crxde/logs?tail=1000
This seems to no longer be possible in AEM 6.2, this path does not resolve to anything.
Is there another way I could still tail the log over HTTP?
A colleague answered this question for me on a chat so I'm putting it here to make it easier to find in the future.
There's now a neat utility in the OSGi console that allows one to view the logs as well as configure the various loggers. You can find it at http://localhost:4502/system/console/slinglog
The Appender tab provides links to the various log files that can be used to load logs over HTTP.
Here's an example request it makes:
http://localhost:4502/system/console/slinglog/tailer.txt?tail=1000&name=%2Flogs%2Ferror.log
As you can see, both the log file name and the tail parameter can be specified. You can also use grep with both simple phrases and regular expressions.
This is a built-in feature of Apache Sling.
In addition FYI, you can also find the status-slinglogs where you can perform log file downloads in a zip and logger actions in a txt to your local at /system/console/status-slinglogs
http://localhost:4502/system/console/status-slinglogs
and the direct urls for the downloading these zip files are as below
http://localhost:4502/system/console/status-slinglogs.zip
http://localhost:4502/system/console/status-slinglogs/configuration-status-20170126-183246.zip (where as 20170126-183246 is and time stamp)
You should not be looking at log files via CRXDE lite.
log files in 6.2 are project specific - better to open them from a text editor.
see attached screenshot.
Hope this helps!
Regards,
Prince
You can curl the log with e.g.:
curl -u admin:admin 'http://localhost:4502/system/console/slinglog/tailer.txt?tail=4000&name=%2Flogs%2Ferror.log'
where 4000 is the number of lines you want to get.
I recently wrote a tool named "Log Tailer Plus" to solve exactly this problem. It's entirely free/open source - Take a look at a post describing usage here : https://blogs.perficientdigital.com/2019/05/14/introducing-aem-logtailerplus/
TLDR; You can grab an AEM package from here ( https://github.com/prftryan/LogTailerPlus ) install it to your machine, and access via http://localhost:4502/log-tailer-plus (if local) or http://server:port/log-tailer-plus
This tool will allow you to follow any number of logs at once by leveraging the out of the box logging endpoint(/system/console/tailer ) as well as dynamically checking active OSGI Logging Logger configurations. Currently, highlighting is supported, but only for relatively standard logging patterns (it's done via regex).
This is a new release, works on AEM 6.2+. Enjoy
I'm working on setting up and migration of old sites to a new server from Dreamhost. I have 130 sites to migrate. 1 is successful. The version is: 1.12.1
The mods in the first site upgraded well. No problems. Have a procedure to migrate. That being said, the second site, as I followed my own documentation, fails when I try to access /admin/index.php
What I get is this below:
So, this is more informational than code... so please forgive me. I don't understand why CMS MADE SIMPLE is actually not simple.
That seems like an annoying issue. Forgive any repetition but here is what I would check:
that is is 100% functioning on existing server
correct php version 5.4.3+ (5.6+ recommended) on new server
that all files & database are copied fully & without errors
config.php settings are updated for new hosting & database
that .htaccess, php.ini/.user.ini settings are appropriate
check php error log
are there any additional modules installed that may require additional php modules
try more coffee or a nights sleep - both have helped me solve all sorts of issue in the past!
consider posting on CMSMS forum - will get a wider range of CMSMS users/experience and suggestions
Good luck
Chris
Not sure what process you are using. So presuming that the site was working correctly on existing hosting and that the new hosting meets recommended requirements, especially PHP version:
Export database
Zip/Compress all website files into a single file
Copy zipped file to new hosting
Create new database & user with full access
Unzip files and make sure that they are in the right location (probably website root)
CHMOD config.php to 0644 and edit database, username & password settings for new hosting, CHMOD back to 0444
Make sure .htaccess is using correct settings for new hosting
Login to admin and clear cache
Sorry that the instructions are so basic, but the process really is Simple.
Possible issues can occur if:
the PHP version is older than 5.4.3 (5.6+ recommended).
files copied individually using ftp and some are corrupted/not copied.
Apart from that it is pretty straight forward.
Hope it helps
I have downloaded and started the community version of SFSx2. I read everything on their documentation page:
http://docs2x.smartfoxserver.com/DevelopmentBasics/introduction
Which unfortunately only talks about flash client side code. Nothing about the corresponding server side code, nor about html5 client side which I need.
I downloaded the HTML5 examples, which took me a long time to find (they are here: http://www.smartfoxserver.com/download/get/140 )
None of the examples work, as they can't connect to the server. Presumably, this is because the examples only supply client side code. There are no instructions supplied on how to install or run the examples.
I can find no mention on the smartfox documenation on the following:
1) what language is used on the server side. One assumes its java.
2) how does one deploy java code to the smartfox server? I cant find any mention of this in the docs.
3) how does one find and install the server part of the client side examples (which are Tris, GameLobby, BuddyMessenger, AdvancedChat).
I applied to be allowed to post on the smartfoxserver forums, but no reply yet.
I also found it hard so I share what I've found out.
The server extensions are written in Java. I used Eclipse JUNO to write my code.
You can download Eclipse here. http://www.eclipse.org/downloads/
You export the extension in jar format from eclipse into you extension path. The name of your file has to end in 'extension' eg MyFirstExtension.jar otherwise sfs2x wont recognize it. Your extension path will be something like this: C:\Program Files\SmartFoxServer2X\SFS2X\extensions\MyFirstExtension\MyFirstExtension.jar if your working in windows.
You will find docs on JAVA API here. http://docs2x.smartfoxserver.com/api-docs/javadoc/server/
This is a link to the basic example code: http://www.smartfoxserver.com/download/get/120
Unzip the content into the [SFS2X_install_folder]/SFS2X/www/root/examples folder, overwriting the existing file. Run the index.html file, then navigate to another index.html to open the example. Run the sfs2x-standalone.exe first of course(see below). If you followed the 'server configuration tutorial' on smartfox website and changed the server ports, the examples wont work. Leave the ports alone until you start to understand the server.
This is a link to the flash example code: http://www.smartfoxserver.com/download/get/108
You will find the example code for apps mentioned above. They are written in Adobe Flash Builder and Java extensions. I don't know if there is any code for HTML5 but the Java server code and AS3 is there.
I also found that using the standalone server was the way to go rather than using SFS2x as a service. c:\program files\smartfoxserver2x\sfs2x\sfs2x-standalone.exe. Using the exe gives you the command prompt window. You can use the window for debugging and see the state of the server when it starts.
Most of the examples should work without creating extensions but to create an extension out of source examples:
Create and setup a new project in your Java IDE. You will have already set up a workspace.
Copy the content of the /source/server/src folder to your project's source folder.
Add SFS2X Libraries. Go to properties of the project -> Select Java Build Path -> Click Library Tab -> Add external jar. Add 'sfs2x.jar' and 'sfs2x-core.jar' from C:\Program Files\SmartFoxServer2X\SFS2X\lib folder. To create the extension, export jar file to extension path. Restart server.
Another problem I had was the Java Version I was using. I had to use Java1.6 with my version of sfs2x when writing extensions. This was trial and error as there was no documentation. There may be a newer version out now.
I had to copy all the server source into eclipse to try and understand how things were done. It was a way of having all the code in one place. There was a lot of trial and error as getting help is hard. I eventually accomplished what I set out to achieve. Good Luck.
SmartFox Server is easy to use even with extension. documentation Give try to Nuggeta solution for game development too. No extension needed at first. This is optionnal.
We have full HTML5 open source game walkthrough on github
So I am attempting to modify an application written by another programmer. The program is written in Perl and apparently uses the Catalyst framework neither of which I have any experience with.
The code is well documented and my modifications seem pretty straightforward however when I try to change something (in the the controllers to be specific) the same to take no effect. Am I missing a step? I open the file edit it, save it, and try to load the web app in my browser. I've even deleted the entire contents of one of the controllers to see if it would break the application and it did not.
Please Help.
Thanks,
Ken
If the application was set-up in a sane way (using uri_for(_action) in templates and not specifically relying on the server/env/etc) you should be developing with the dev server. There are some practices that can make this difficult:impossible without modifications. This is all you should have to do–
cd {APPLICATION DIRECTORY}
# Read about it-
perldoc script/*_server.pl
# Run it-
script/*_server.pl -r -d
Unless there is something wonky in the setup, you’ll get http://localhost:3000/ running with your app.
Or, what is probably a good idea, run the application as the webuser in your apache setup. If there are files or access expected to be for that user, it might be important (e.g., if session or cache files are used and restricted to the user)–
sudo -u www script/*_server.pl -r -d
The flags turn on debugging output and the restarter so that every time you change files in the application, the server will restart automatically (if it compiles).
Catalyst is a joy to develop with and the dev server is part of why.