I am trying to do an non-recursive list using gcloud-java storage on GCS bucket outside appengine based on this suggestion
However it seems that its not working as expected, namely it is still listing all the files although I have set the Storage.BlobListOption.recursive(false)
my code structure is basically like:
Storage client = StorageOptions.defaultInstance().service();
Page<Blob> blobPage = client.list(my_bucket, Storage.BlobListOption.recursive(false));
for (Blob blob : blobPage.values()) {
// it is still iterating all the leaf files in my_bucket
}
Actually later on I found that it seems that none of BlobListOption seems to be taking affect (maxResults, prefix) and list will just return all leaf files no matter what...
Am I using the API wrong or what might be the problem?
There was a bug with non-recusrive blob and prefix (folder) listing.
This was fixed in since v 0.1.15 (see this). The recursive option
was replaced with currentDirectory and now you can get the prefixes/child
folders. If this does not work for you or you think any of the other options does not work as intended please file an issue and it will be
taken care of quickly.
Related
Years ago, I wrote a small app in itext2 to gather reports on a weekly basis and concatenate them into one PDF. The app used com.lowagie.text.pdf.PdfCopy to copy and merge the PDFs. And it worked fine. Performed exactly as expected.
A few weeks ago I looked into migrating the application to itex7. To that end, I used the copyPagesTo method of com.itextpdf.kernel.pdf.PdfDocument. When run on the same file set, this produces warnings like:
WARN PdfNameTree - Name "section.1" already exists in the name tree; old value will be replaced by the new one.
When I click on the link to "section.1" in the first document of the merged PDF, I am taken to "section.1" of the last document. Not what I expected and not what happens when using the itext2 app. In the PDF's produced by itext2, if I click on the link to "section.1" of the first document in the combined PDF, I am taken to section 1 of the first document.
There is a hint in Javadocs for copyPagesTo saying
If outlines destination names are the same in different documents, all
such outlines will lead to a single location in the resultant
document. In this case iText will log a warning. This can be avoided
by renaming destinations names in the source document.
There is however, no explanation of how this should be done. I find it odd that this should be necessary in itext7, although it wasn't in itext2.
Is there a simple way to get around his problem?
I've also tried the Sejda desktop app and it produces correct results, but I would prefer to automate the process through a batch script.
My guess is iText 2 didn't even know it might be a problem.
If iText can't deduplicate destination names, the procedure is roughly:
Follow /Catalog -> /Names -> /Dests in each document to find the destination name tree.
Deduplicate the names, by adding suffixes. Remember that a name with a suffix added might be equal to an existing name in the same or another document. Be careful!
Now you can rewrite the destination name trees. Since you have only used suffixes, you can do this in place - the lexicographic ordering of the names is unaltered so the search tree structure is not broken.
Now, rewrite destination links in each PDF for the new names. For example any dictionary entry with key /Dest, or any /D in a /GoTo action.
Now, after all this preprocessing, the files will merge without name clashes.
(I know all this because I've just implemented it for my own PDF software. It's slightly hairy stuff, but not intractable.)
If you like, I can provide a devel version of cpdf with this functionality, if you would like to test it.
I rely heavily on the File: Open Recent… command to open frequently used files, but yesterday my local Google Drive folder got moved to a new location and now I can no longer access any of the files in that folder through the Open Recent panel because the paths don't match.
The fix would be as simple as replacing "/Google Drive/" with "/Google Drive/My Drive/" but I have no idea what file contains the list of files that appears in the recently opened panel.
I'm assuming it's somewhere in ~/Library/Application Support/Code but not sure where.
I was wondering the same thing the other day and found this while searching for a solution, so I took some time to investigate it today.
It's been a a few weeks since you posted, so hopefully this will still be of help to you.
Also, I'm using Windows and I'm not familiar with macOS, but I think it should be easy enough adjust the solution.
Location of settings
Those setting are stored in the following file: %APPDATA%\Code\User\globalStorage\state.vscdb.
The file is an sqlite3 database, which is used as a key-value store.
It has a single table named ItemTable and the relevant key is history.recentlyOpenedPathsList.
The value has the following structure:
{
"entries": [
{
"folderUri": "/path/to/folder",
"label": "...",
"remoteAuthority": "..."
}
]
}
To view the current list, you can run the following command:
sqlite3.exe -readonly "%APPDATA%\Code\User\globalStorage\state.vscdb" "SELECT [value] FROM ItemTable WHERE [key] = 'history.recentlyOpenedPathsList'" | jq ".entries[].label"
Modifying the settings
Specifically, I was interested in changing the way it's displayed (the label), so I'll detail how I did that, but it should be just as easy to update the path.
Here's the Python code I used to make those edits:
import json, sqlite3
# open the db, get the value and parse it
db = sqlite3.connect('C:/Users/<username>/AppData/Roaming/Code/User/globalStorage/state.vscdb')
history_raw = db.execute("SELECT [value] FROM ItemTable WHERE [key] = 'history.recentlyOpenedPathsList'").fetchone()[0]
history = json.loads(history_raw)
# make the changes you'd like
# ...
# stringify and update
history_raw = json.dumps(history)
db.execute(f"UPDATE ItemTable SET [value] = '{history_raw}' WHERE key = 'history.recentlyOpenedPathsList'")
db.commit()
db.close()
Code references
For reference (mostly for my future self), here are the relevant source code areas.
The settings are read here.
The File->Open Recent uses those values as-is (see here).
However when using the Get Started page, the Recents area is populated here. In the Get Started, the label is presented in a slightly different way:
vscode snapshot
The folder name is the link, and the parent folder is the the text beside it.
This is done by the splitName method.
Notes
Before messing around with the settings file, it would be wise to back it up.
I'm not sure how vscode handles and caches the settings, so I think it's best to close all vscode instances before making any changes.
I haven't played around with it too much, so not sure how characters that need to be json-encoded or html-encoded will play out.
Keep in mind that there might be some state saved by other extensions, so if anything weird happens, blame it on that.
For reference, I'm using vscode 1.74.2.
Links
SQLite command-line tools
jq - command-line JSON processor
Update: Check my answer below.
I just realized that in Jekyll Webrick server, directories starting with underscores(_includes, _layouts etc.) can't be accessed and are not listed when jekyll serve --show-dir-listing option is turned on. I wonder how Jekyll does that, as Webricks shows underscored directories on default. I did a quick search in the source code, I checked lib/jekyll/commands/serve.rb and similar files, but could not find the exact reason. It might be something related to fancy_listing?
Example:
It is there!:
Update: I found the relevant code in jekyll/reader.rb, which has a filter function and it is defined in jekyll/entry_filter.rb! :) Here is the code:
First a regex is defined:
SPECIAL_LEADING_CHAR_REGEX = %r!\A#{Regexp.union([".", "_", "#", "~"])}!o.freeze
Then special?function is defined:
def special?(entry)
SPECIAL_LEADING_CHAR_REGEX.match?(entry) ||
SPECIAL_LEADING_CHAR_REGEX.match?(File.basename(entry))
end
And special?function is used in the filter function to detect and filter those files matching the regex.
And the Readerclass is using this filtering function in various places.
To be honest, I still did not get how jekyll bring those things together but I think I'll try to figure them out myself.
I'm seeing some very strange behavior with FileMaker 14. I'm using LayoutObjectNames for some required functionality. On the development system it's working fine. It returns the list of named objects on the layout.
I close the file, zip it up and send it to the client, and that required functionality isn't working. He sends the file back and I open it and get a data viewer up. The function returns nothing. I go into layout mode and confirm that there are named objects on the layout.
The first time this happened and I tried recovering the file. In the recovered file it worked, so I assumed some corruption had happened on his end. I told him to trash the file I had given him and work with a new version I supplied. The problem came up again.
This morning he sent me the oldest version that the problem manifested in. I confirmed the problem, tried recovering it again, but this time it didn't fix the problem.
I'm at a loss. It works in the version I send him, doesn't on his system. We're both using FileMaker 14, although I'm using Advanced. My next step will be to work from a served file instead of a local one, but I have never seen this type of behavior in FileMaker. Has anyone seen anything similar? Any ideas on a fix? I'm almost ready to just scrap the file and build it again from scratch since we're not too far into the project.
Thanks, Chuck
There is a known issue with the Get (FileName) function when the file name contains dots (other that the one before the extension). I will amend my answer later with more details and a possible solution (I have to look it up).
Here's a quote from 2008:
This is a known issue. It affects not only the ValueListItems()
function, but any function that requires the file name. The solution
is to include the file extension explicitly in the file name. This
works even if you use Get (FileName) to return the file name
dynamically:
ValueListItems ( Get ( FileName ) & ".fp7" ; "MyValueList" )
Of course, this is not required if you take care not to use period
when naming your files.
http://fmforums.com/forums/topic/60368-fm-bug-with-valuelistitems-function/?do=findComment&comment=285448
Apparently the issue is still with us - I wonder if the solution is still the same (I cannot test this at the moment).
Apologies for the lack of precision in the question, but I'm not completely sure which of possibly many things I'm doing wrong here.
I'm relatively new to Coffeescript and also geo applications in general, but here goes:
I've got a working (simple) Meteor (.7.0.1) application utilizing coffeescript in both client and server. The issue I'm having occurs when attempting to utilize TopoJSON encoded files to create a layer of US congressional districts. (the purpose of the app is to help highlight voter suppression in the US)
So, a few things: Typically in a non-Meteor app, I would just load the topoJSON file like so:
$.getJSON('./data/us-congress-113.json', function (data) {
var congress_geojson = topojson.feature(data, data.objects.districts);
congress_layer.addData(congress_geojson);
});
Now of course this won't work in Meteor because its not asynchronous.
One of the things that was recommended here on SO was to not worry about reading the file, and to instead change the json file to .js, and then set the contents (which are of course just an object) equal to a variable.
Here's what I did:
First, I changed the .json file to a .js file in the server directory, and added the "congress =" to the beginning of the file. It's a huge file so forgive me for omitting the whole object.
congress = {"type":"Topology",
"objects":
{"districts":
{"type":"GeometryCollection","geometries":[{"type":"Polygon"
Now here's where everything starts to give me issues:
In the server.coffee, I've created a variable like so to reference the congress object:
#congress_geojson = topojson.feature(congress, congress.objects.districts)
Notice how I'm putting the # symbol there? I've been told this allows a variable in Coffeescript to be globally scoped? I tried to also use a Meteor feature called "share" where I declare the variable as "share.congress_geojson". That led to the same issues which I will describe below.
Now in the client.coffee file, I'm trying to call this variable to load into a Leaflet map.
congress_layer = L.geoJson(null,
style:
color: "#DE0404"
weight: 2
opacity: 0.4
fillOpacity: 0.1
)
congress_layer.addData(#congress_geojson)
This isn't working, and specifically (despite attempts to find other ways, the errors I'm getting in the console are:
Exception from Deps afterFlush function: TypeError: Cannot read property 'features' of undefined
at o.GeoJSON.o.FeatureGroup.extend.addData (http://localhost:3000/packages/leaflet.js?ad7b569067d1f68c7403ea1c89a172b4cfd68d85:39:16471)
at Object.Template.map.rendered (http://localhost:3000/client/client.coffee.js?37b1cdc5945f3407f2726a5719e1459f44d1db2d:213:18)
I have no doubt that I'm missing something stupidly obvious here. Any suggestions or tips for what I'm doing completely wrong would be appreciated. Is it a case where an object globally declared in a .js file isn't available to code in a .coffee file? Maybe I'm doing something wrong on the Meteor side?
Thanks!
Edit:
So I was able to get things working by putting the .js file containing the congress object in a root /lib folder, causing the object to load first, and then calling the congress object from the client. However, I'm still wanting to know how I could simply share this object from the server? What is the "Meteor way" here?
If you are looking for the Meteor way to order the loading of files, use the Meteor.startup function and put the initialization code there. That function is the $.ready of the Meteor world, i.e., it will execute only after all your files have been successfully loaded on the client.
So in your case:
Meter.startup ->
congress_layer.addData(#congress_geojson)