I've been using Magmi with great success, creating and updating our magento products on a daily basis.
Our production retail site generally uses the default/admin values for store. When I make new categories and populate them I generally use the category_reset=0 column to preserve the handmade sort order or position values for all of the original categories.
I've been working on a wholesale site set up with a seperate filesystem for all 3 levels of the Magento hierarchy. I did an import with magmi setting the store column to the wholesale site, with 2 additional collumns - sku and category_ids (without category_reset) using a sub-set of data exported from the admin store view (filtered the manufacturer column for only one manufacturer) to try to populate the wholesale site categories (same root catalog with certain categories disabled or not visible) with the same category products.
For some reason, I'm not sure why, (ouch, I realize now there was a typo in the header name for store) it did not update the right store - it defaulted back to admin and lost
the sort order for many categories, about 3k products imported ok.
I have 2 non-production sandbox sites with duplicate category data. I've been manually copying the category product listings with the desired position values into a new csv so I will have sku,category_id (singular),position_value
Many products are members of more than one category. My question is...
In order to regain the position values or sort order, what syntax should I use under category_ids? The products are already in the category so I would use a category_reset=0 column, right?
for an example record:
sku category_ids
45000 39,262,353
my next import might look like:
sku category_ids category_reset
abc 39::10 0
def 39::20 0
45000 39::30 0
ghi 262::10 0
45000 262::20 0
jkl 262::30 0
45000 353::10 0
mno 353::20 0
does this seem workable? I'm feeling very gunshy after having borked my production site with a typo and need some validation before I take steps to confuse myself further.
Thanks in advance for any insight.
As stated in the Magmi Documentation for Importing item positions in categories (from magmi version 0.7.18), the syntax is as follows:
sku,....,category_ids
000001,...,"8::1" < = put sku 00001 at position 1 in category with id 8
000002,...,"9::4,7" < = put sku 00002 at position 4 in category with id 9 and at position 0 in category with id 7
000003,...,"8::10" <= put sku 00002 at position 10 in category with id 8
So yes, your method should work. Be sure to do a full database backup before doing major import changes ;)
Related
I have three tables:
(1) Audit Findings [up to 100 records per audit, depending on the standard],
(2) Types of Documents to check [different types and number, depending on the standard against which an audit is conducted], and
(3) Names of the documents that need to be present.
Records of (1), where the main audit findings are entered, contain a portal that lists the required documents. This list is provided by table (2). Next to this portal list, a "Document Type" relationship between (2) and (3) ensures that the correct document names appear after the relevant document types. Example: Shipping Notes: SN2234, SN8926; Sales Invoices: IV5673, IV7251, etc.
I now need to link the document names of table (3) to the audit at hand, i.e. table (1), to avoid that audit findings for any company always list the same ported document names. In other words, the 2 = 3 relationship needs to be filtered based on the audit date (clients are audited once a year), client number and standard info (most clients are multi-cetified) contained in table (1). Is this possible? And how?
This is more of a guess than an answer. It is based on the following assumptions:
Three tables, related as:
AuditFindings --< DocumentTypes --< DocumentNames;
There is a portal to DocumentNames, placed on a layout of AuditFindings, showing all the DocumentNames "grandchildren" of the currently viewed record in AuditFindings.;
There is a ActiveDate field in AuditFindings table;
There is a DocumentDate field, in the DocumentNames table.
Now, in order to filter the portal mentioned in point #2 above, so that it shows only records of matching dates, set the portal filter to show records only when:
AuditFindings::ActiveDate = DocumentNames::DocumentDate
My CMS is currently Movable Type 5.04. The attached screenshot is how I check in Compose Screen of MT CMS.
And it appears in the new entry create page with the following order.
- Location 5
- Job Description 5
- Bio 5
- Job Title 5
I would like to change the order into following.
- Job Title 5
- Location 5
- Job Description 5
- Bio 5
Is there any way to do so? I have checked in the cfg_entry.tmpl file and the above Entry fields were not there as there were custom fields. Could you please help me sort them like above?
the order of the fields are stored into the permission table, in 'permission_entry_prefs' field. (and there is a matching field for the page)
the row with blog_id 0 is the user defaults, and each blog's raw will override the default for this blog.
and the format is very simple: comma delimited list. you should find there something like "title,text,tags,assets". just reorder them for your liking.
If the user will toggle on and off fields, I'm not sure if the original order will return or not. if it does, then a simple plugin can solve it in more permanent way.
I remember doing this by recreating the custom fields in the wanted order (MT pulls them from the DB in a specific order, sorted by ID IIRC). If you have existing data, then you need to delete the fields definitions (not the data) then recreate them in the order you want changing just their numerical ID (not anything else).
I'm traveling and busy at the moment so can't dig into details. But study how CF are defined in the DB and how to simply reorder them by ID.
I need to do the stock management website with 2 Content type Supplier and Product. First I add contents in Supplier content type. Then I add node reference field in Product content type and call it supplier and make it multiple value. it's looks good, i can select suppliers and save it. but in my concept i need to check which supplier sale this product and how price. so i need to add suppliers and price in the same form like my image.
the propose of this form, user can check how price of each supplier for this product and they can choose the lowest price for purchase in the next process.
Guys, did you have the idea that i can do like this ?
Based on your image attached, I'd assume you have Druapl 7.
You can do this by installing Field Collection module. To get the table-input, install Field Collection Table.
Core Fields can have multiple instances of the same field. But you can't group them and make the whole group a multi-instance group.
Drupal 6 required CCK module to have fields, and there was a MultiGroup module (CCK 3branch which never had a stable release) that does the similar for Drupal 6.
Although OP will not need this, dear Googler if you are looking for a simple table with text fields, try TableField module
Hi all and thanks in advance.
I am new to the NoSQL game but my current place of employment has tasked me with set comparisons of some big data.
Our system has customer tag set and targeted tag sets.
A tag is an 8 digit number.
A customer tag set may have up to 300 tags but averages 100 tags
A targeted tag set may have up to 300 tags but averages 40 tags.
Pre calculating is not an option as we are shooting for a potential customer base of a billion users.
(These tags are hierarchical so having one tag implies that you also have its parent and ancestor tags. Put that info aside for the moment.)
When a customer hits our site, we need to intersect their tag set against one million targeted tag sets as fast as possible. The customer set must contain all elements of the targeted set to match.
I have been exploring my options and the set intersection in Redis seems like it would be ideal. However, my trolling through the internet has not revealed how much ram would be required to hold one million tag sets. I realize the intersection would be lightning fast, but is this a feasable solution with Redis.
I realize this is brute force and inefficient. I also wanted to use this question as means to get suggestions for ways this type of problem has been handled in the past. As stated before, the tags are stored in a tree. I have begun looking at Mongodb as a possible solution as well.
Thanks again
This is an interesting problem, and I think Redis can help here.
Redis can store sets of integers using an optimized "intset" format. See http://redis.io/topics/memory-optimization for more information.
I believe the correct data structure here is a collection of targeted tag sets, plus a reverse index to map tags to their targeted tag sets.
To store two targeted tag sets:
0 -> [ 1 2 3 4 5 6 7 8 ]
1 -> [ 6 7 8 9 10 ]
I would use:
# Targeted tag sets
sadd tgt:0 1 2 3 4 5 6 7 8
sadd tgt:1 2 6 7 8 9 10
# Reverse index
sadd tag:0 0
sadd tag:1 0
sadd tag:2 0 1
sadd tag:3 0
sadd tag:4 0
sadd tag:5 0
sadd tag:6 0 1
sadd tag:7 0 1
sadd tag:8 0 1
sadd tag:9 1
sadd tag:10 1
This reverse index is quite easy to maintain when targeted tag sets are added/removed from the system.
The global memory consumption depends on the number of tags which are common to multiple targeted tag sets. It is quite easy to store pseudo-data in Redis and simulate the memory consumption. I have done it using a simple node.js script.
For 1 million targeted tag sets (tags being 8 digits numbers, 40 tags per set), the memory consumption is close to 4 GB when there are very few tags shared by the targeted tag sets (more than 32M entries in the reverse index), and about 500 MB when the tags are shared a lot (only 100K entries in the reverse index).
With this data structure, finding the targeted tag sets containing all the tags of a given customer is extremely efficient.
1- Get customer tag set (suppose it is 1 2 3 4)
2- SINTER tag:1 tag:2 tag:3 tag:4
=> result is a list of targeted tag sets having all the tags of the customer
The intersection operation is efficient because Redis is smart enough to order the sets per cardinality and starts with the set having the lowest cardinality.
Now I understand you need to implement the converse operation (i.e. finding the targeted tag sets having all their tags in the customer tag set). The reverse index can still help.
Here in an example in ugly pseudo-code:
1- Get customer tag set (suppose it is 1 2 3 4)
2- SUNIONSTORE tmp tag:1 tag:2 tag:3 tag:4
=> result is a list of targeted tag sets having at least one tag in common with the customer
3- For t in tmp (iterating on the selected targeted tag sets)
n = SCARD tgt:t (cardinality of the targeted tag sets)
intersect = SINTER customer tgt:t
if n == len(intersect), this targeted tag set matches
So you never have to test the customer tag set against 1M targeted tag sets. You can rely on the reverse index to restrict the scope of the search to an acceptable level.
this might be helpful:
Case Study: Using Redis intersect on very large sets (120M+ with 120M+)
http://redis4you.com/articles.php?id=016&name=Case+Study%3A+Using+Redis+intersect+on+very+large+sets
The answers provided helped me initially. However as our customer base grew, I stumbled across a great technique involving using redis string bits and bit operators to perform analytics on hundreds of millions of users very quickly.
Check this article out. Antirez, creator of redis, also references this a lot.
http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/
I'm working on an iPhone app with a GAE backend. I currently have a database of ~8000 products and each product has 5 keywords, mined from reviews, that are the words used most often to describe the product. Once I deploy the app, I'd like to allow users to add new products, and add their 5 keywords to existing products. So, when "reviewing" an existing product, they would add their 5 words, and these would be reflected in the Top 5 words if they push a word over into the Top 5. These keywords will be selected via a large whitelist with indirect selection so I can control the user input. I'd like the application to scale to thousands of users without hitting my backend too hard.
My question is:
What's the most efficient database schema for keeping track of all the words for a product and calculating the top 5 for each product once it's updated?
My two ideas (which may be terrible):
Have a "words" column which contains a 2d array, one dimension is the word, the other is the count for that word. They would then be incremented/decremented as needed.
Have a database with each word as a column and each product as a row and the corresponding row/column would contain the count.
The easiest way to do this would be to have a 'tags' kind, defined something like this (you haven't specified a backend language, so I'm assuming Python):
class Tag(db.Model):
# Tags should be child entities of Products and have key name based on the tag
# eg, created with Tag(parent=a_product, key_name='awesome', ...)
count = db.IntegerProperty(required=True, default=0)
#classmethod
def increment_tags(cls, product, tag_names):
def _tx():
tags = cls.get_by_key_name(tag_names, parent=product)
for i, tag in enumerate(tags):
if tag is None:
# New tag
tags[i] = tag = cls(key_name=tag_names[i], parent=product)
tag.count += 1
db.put(tags)
return db.run_in_transaction(_tx)
#classmethod
def get_top_product_tags(cls, product, num=5):
return [x.key().name() for x
in cls.all().ancestor(product).order('-count').fetch(num)]
The increment_tags method increments the count property on all the relevant tags. Since they all have the same parent entity, they're in the same entity group, and it can do this transactionally, in a single transaction.
The get_top_product_tags method does a simple datastore query to find the num top ranked tags for a product.
You should use a normalized schema and let SQL and the database engine be your friend. Have a single table with a design like this:
create table KeywordUse
( AppID int
, UserID int
, Sequence int
, Word varchar(50) -- or whatever makes sense
)
You can also have an identity primary key if you like, but AppID + UserID + Sequence is a candidate key (i.e. the combination of these three must be unique).
To find the top 5 keywords for any app, do a SQL query like this:
select top 5
count(AppID) as Frequency -- If you have an identity PK count that instead.
, Word
from KeywordUse
where AppID = #AppIDVariable...
group by Word, AppID
order by count(AppID) desc
If you are really, really worried about performance you could denormalize the results of this query into a table that shows the words for each app. Then you'd have to work out how often to refresh that snapshot.
REVISED ANSWER:
As Nick Johnson so generously pointed out, aggregate functions are not available in GQL. However, the philosophy of my answer remains unchanged. Let the database engine do its job.
The table should be AppID, Word, and Frequency. (AppID and Word are the PK.) Then each use of the word would be added up as it is applied. Then, when you want to know the top five words for an app you select by AppID := #Value and order by Frequency (descending) with a LIMIT = 5.
You would need a separate table to track user keywords if that is important.