I have, in order to process some big data, to set up ckan on a local machine. I've set up the whole system following this guide : http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html
I wanted to display a preview of a locally loaded file, so the user can actually see it before downloading it. And it doesn't work, because it only works for online files. For instance, it DOES work with this online file but NOT with my own file I upload.
So, I've been interested about Datastore and Datapusher. I've followed every part of the guide, and it appears on my ckan. However, I have an error. Specifically this one :
Upload error: An Error occurred while sending the job: 403 Client Error: Forbidden for url: http://127.0.0.1:8800/job
Here's my most important parts about my production.ini file (copying the whole would be very long) :
ckan.site_url = http://localhost
ckan.plugins = datastore datapusher stats text_view image_view
recline_view recline_graph_view recline_map_view webpage_view
ckan.datapusher.formats = csv xls xlsx tsv application/csv
application/vnd.ms-excel
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
ckan.datapusher.url = http://127.0.0.1:8800/
I truly have no idea about what my problem could be, I tried to change the datapusher.url to 0.0.0.0 as the guide suggested, but it doesn't work either.
If the data to be added to CKAN is in a file on your computer, select “Upload a file” option. CKAN will give you a file browser to select it. You should use link to a file option just for publicly available resources.
Have you installed datapusher also? Its a separate process running on port 8800. CKAN uses datastore to be able to have a grid view of tabular data. Data needs to be pushed through datapusher to be used by datastore.
Yes, you need to set up the Datapusher.It's not activated by default.
Pull the datapusher code, install the dependencies and run it using:
python datapusher/main.py deployment/settings.py
The instructions to configure the settings are on the repository.
Here's the datapusher manual: http://docs.ckan.org/projects/datapusher/en/latest/
Here's the repository: https://github.com/ckan/datapusher
Had the exact same error message.
This post solved my issue though.
short: insert/check the following in your virtualhost in /etc/apache2/sites-enabled/datapusher.conf
<Directory /etc/ckan>
Options All
AllowOverride All
Require all granted
</Directory>
Related
In the AWS documentation for "Connecting to your DB instance using IAM authentication and the AWS SDK for Python (Boto3)", the following call is made to both psycopg2.connect (shown) and mysql.connector.connect:
conn = psycopg2.connect(host=ENDPOINT, port=PORT, database=DBNAME, user=USR, password=token, sslmode='prefer', sslrootcert="[full path]rds-combined-ca-bundle.pem")
cur = conn.cursor()
cur.execute("""SELECT now()""")
query_results = cur.fetchall()
print(query_results)
I see some discussion about the ssl_ca path (here and here) and what those bundles are used for. But none of the three links I've given here describe the [full path] component given by the AWS docs, or where it is pointing to. My current guess (from the second link) is this URL, but I'd like to be sure.
Additionally, what are the advantages to having this bundle downloaded to the remote EC2 on which these Python 3 (boto3) scripts are running?
EDIT: By the way, the above call to psycopg2.connect is working in Jupyter with Python 3.9.5 on an EC2 currently, with the [full path] written as-is...
You should replace the '[full path]' with the filesystem path (directory path) to where you saved the pem file when you downloaded it (from that last URL you gave) to the local computer.
The advantage of using it is that your client will verify it connected to the correct database, and not some malicious system which is intercepting your traffic. I don't how advantageous you consider this: if someone has compromised Amazon enough to be intercepting their internal traffic, they might also have compromised their CA as well. But there is at least some possibility they did one without the other.
Your code as shown does not work for me, because ssl_ca is not how it is spelled. Assuming you used the code actually given at your first link for PostgreSQL:
sslmode='prefer', sslrootcert="[full path]rds-combined-ca-bundle.pem"
Then the reason it works despite the bogus path is that "prefer" means it doesn't care if the rootcert is missing, it just skips validating in that case. If you change it to 'verify-full', then presumably it would stop working.
I am using Aspera Connect on mac to download files from a server. It works fine in terminal, but i was wondering if before i download a file, i could read its size first and then decide if i want to download it or not. I found the flag
'--precalculate-job-size'
but it's only doing that right before download and there's no way to stop the download.
The current command i use is this:
/Applications/Aspera\ Connect.app/Contents/Resources/./ascp -QT -l 200M -P33001 -i "/Applications/Aspera Connect.app/Contents/Resources/asperaweb_id_dsa.openssh" emp_ext3#fasp.ebi.ac.uk:/{asp_path} {local_path}
The resources for the flags are here:
https://download.asperasoft.com/download/docs/ascp/2.7/html/index.html
To answer your question, without going too much in the details:
If you want to display the size of an elements on an Aspera server for which you have access, you can use the command line "Amelia", see:
https://www.rubydoc.info/gems/asperalm
mlia server --url=ssh://fasp.ebi.ac.uk:33001 --username=emp_ext3 --ssh-keys=~/.aspera/mlia/aspera_bypass_dsa.pem br /10002/data/100_movie_gc.mrcs
there are plenty of options, like : --format=csv --fields=size
Note that this displays individual file sizes, but not recursive folder size.
a few other things:
You are not exactly using "Connect", but rather the "ascp" command line. Connect refers rather to the browser extension and lightweight app. while ascp is the implementation of Aspera FASP transfer protocol, found basically in all Aspera products.
the latest ascp documentation can be found here: https://www.ibm.com/support/knowledgecenter/SSL85S_3.9.6/hsts_admin_linux/dita/hsts_admin_linux_ascp_usage.html
did you know you can also use the free client:
https://downloads.asperasoft.com/en/downloads/2
it includes also ascp, but also a graphical user interface
I have a ZF3 project with a controller which opens excel-files and compares them with an template which will be openened, too.
On my development notebook (xampp) everything works fine, at my production system (ubuntu) the phpspreadsheet causes errors (I think it is the one).
here a snippet from my code:
$fileName="./public/files/" . $fileName; //.\ neu
echo $fileName;
$template= new Spreadsheet();
$importdcl= new Spreadsheet();
$template= \PhpOffice\PhpSpreadsheet\IOFactory::load('./public/files/Template_DCL_final.xlsx');
$importdcl= \PhpOffice\PhpSpreadsheet\IOFactory::load( $fileName);
echo "filename geladen";
I already have the folders in non relative paths because basePath() doesn't work, it won't give any result.
The echo statement is just because the server log won't give any errors. On my development system I get the echo text on my production system the error seems to be at the load statements.
First question: How could I use relative paths in here?
Second question: How can I get an idea wether is something wrong with the spreadsheet class?
This is what composer loaded:
"phpoffice/phpspreadsheet" : "dev-develop",
Is it a problem, because it has this dev version? At this point I'm quite confused because I played with the pathes of the files, I changed the rights manually in the folder, I checked server logs and now I don't have any idea left.
Here the rights in the folder:
Any helpful suggestions appreciated.
Answer (hopefully) to the first question: if your application is based on ZendSkeletonApplication, then you can use paths relative to your application root (from index.php):
/**
* This makes our life easier when dealing with paths. Everything is relative
* to the application root now.
*/
chdir(dirname(__DIR__));
So if your data files are located in <application root>/public/files, then you should be able to read them from controller/service/etc. using public/files/<file name> path. You can test it with eg. the file_exists function.
I’m unable to answer your second question, but here are some suggestions (OK, questions…):
what is the status code of your (production) server’s response?
Do you have read/write permissions to the data files from the server’s account (www-data?)?
Does phpspreadsheet depend on any PHP extensions? Do you have them installed on both your development machine and the server?
What PHP version is installed on the server? Do you or phpspreadsheet use any features that may be unavailable on that version?
Try running your project not through Apache, but with PHP’s builtin server (but don’t do it long-term) and try to reproduce the issue.
A copy of a TYPO3 4.4 site has been installed on a Windows 10 system. PHP and Apache versions are approximately equal. The site itself is functional and appears complete. I can log in to the backend of the live site. I cannot log in to the backend of the copied site.
Trials:
Change database character set from latin1 to utf8
Change my credentials to database credentials (username, md5('password')) used in localconf.php
Comment out modified $TYPO3_CONF_VARS['BE']['installToolPassword'] in localconf.php
./typoconf/temp_* deleted
Edit #1:
If localconf.php is modified to point to the live database I am able to log in. This suggests that the issue may be a difference in server versions.
Edit #2:
Installing MySQL server version 5.5.54 (live site has 5.5.53) makes no difference.
Have you tried to set
from:
'loginSecurityLevel' => 'rsa',
to:
'loginSecurityLevel' => 'normal',
in your localconfig.php file?
I had the same problem. The reason was that someone copied the system via phpmyadmin.
I copied the typo3 mysql-DB via
mysqldump ... > ...sql
and
mysql ... < ...sql
and now it works
Maybe you should look at your cookie domain in localconf.php
It took a look a ancient history to see that one source of the error was the MySQL strict mode. The clue that told me I was logged in was a record in the sys_log table with a current timestamp & my userid. An old bug report included a comment on strict mode. By changing MySQL 5.7 my.ini from sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" to sql-mode="NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" I gained access to my sandbox. Hooray for wayback machines everywhere.
Good day everyone, I have been trying to put my moodle online so pcs from internet can access it, but until now, no luck at all. (Im using moodle 2.3.2 on Windows Server 2008 and IIS 7).
I tried to configure the moodle file config.php, setting the directive $CFG -> wwwroot = "my-public-ip/moodle". Then, when I access to moodel from the server, I can access it by "http://my-public-ip/moodle", when I try to access via localhost, it sends an error which it is OK.
But the funny part comes when I try to access the server from an outside pc. When I type "http://my-public-ip/moodle" it simply cant "see" the configuration I made to the config.php file (it says: This server cna only be accessed via localhost/moodle) it looks like the outside pcs are either ignoring it, or searching for another configuration file. I dont know what the hell is happening, this is very odd.
Any ideas?? tnx!!!
Change the following file:
lib-->setuplib.php
Redirect ($CFG->wwwroot, get_string('wwwrootmismatch', 'error', $CFG->wwwroot), 3);
for
Redirect ($CFG->wwwroot, get_string('wwwrootmismatch', 'error', $CFG->wwwroot), 0);
I realise this is an old question, but it's also worth pointing out you may need to also run the database search and replace script, at:
http://my-public-ip/moodle/admin/tool/replace/index.php
as referenced in Method 2 here.
This is required if you change the name of the site once you have installed it. If you were already using Moodle under "localhost", then there will be a number of references to the old localhost address stored in the database that need to be updated to the new IP-based address.
It might be because the http:// part is missing?
$CFG->wwwroot = "my-public-ip/moodle"
should be
$CFG->wwwroot = "http://my-public-ip/moodle"