w3hello.com logo
Home PHP C# C++ Android Java Javascript Python IOS SQL HTML videos Categories
How should I store the data with duplicated keys in a document for MongoDB?
Instead of modeling duplicate fields at the same level in a document, you can use an array of subdocuments for the repeated elements like genes eg: { source: "1..4242774", organism: "Bacillus amyloliquefaciens subsp. plantarum YAUB9601-Y2", mol_type: "genomic DNA", strain: "YAU B9601-Y2", sub_species: "plantarum", db_xref: "taxon:1155777", genes: [ { id: '412..1752', gene: "dnaA", }, { id: '1937..3073', gene: "dnaN", } ], CDS: [ { id: '412..1752', } ] } For more information see Array of Subdocuments in the MongoDB manual.

Categories : Mongodb

How to map document with dynamic keys to a Spring MongoDb entity class
This is one way of doing it. There may be other better ways. Create a map of attributes and store the map in mongo. public class Env { @Id @GeneratedValue(strategy = GenerationType.AUTO) private ObjectId id; @Field private Envinfo envinfo; public static class Envinfo { @Field private String appName; @Field private Map<String, String> attributes; } } If you know the keys in advance, you may add those attributes in Envinfo and keep those out of attributes map.

Categories : Java

ElasticSearch Java API to query and return single column instead of the whole json document
You can specify the fields to be returned from a search, per documentation. This can be set via SearchRequestBuilder.addFields(String... fields), ie: SearchRequestBuilder srb = client.prepareSearch("heros") .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) .addFields("name");

Categories : Java

Not able to delete document from Mongodb database with PHP
_id field is not a string in mongodb, it's an object of ObjectId class; instead of $id = (array("_id" => $_POST["id"])); you should use $id = (array("_id" => new MongoId($_POST["id"])));

Categories : PHP

function to print document in json format (mongodb)
The perl driver will return a cursor when you do find: my $cursor = $collection->find({ i => { '$gt' => 42 } }); You can iterator over this $cursor to obtain the documents, by doing this: while (my $object = $cursor->next) { ... } In the while loop, you can then access each field of each $object: print $object->{i}; If you want, you can also convert them to JSON, but for that you need to have the JSON CPAN module installed. On Debian/Ubuntu, you can use apt-get install libperl-json for that, but add use JSON; to the start of your script. With this installed, you can put in your while clause: while (my $object = $cursor->next) { my $json = encode_json $object; print "$json "; }

Categories : Json

With elasticsearch, it's possible to search document related to another document?
I believe that you're looking for "more like this" querying - here's the ES API documentation: http://www.elasticsearch.org/guide/reference/api/more-like-this/. I'd think that tire exposes this API endpoint or allows you to do so manually - in fact, it looks like tire-contrib give you what you're looking for: https://github.com/karmi/tire-contrib/.

Categories : Search

how to search the database for the keys that are themselves substrings of the search string in mongodb
You can use regex in your search string if it can help: http://docs.mongodb.org/manual/reference/operator/regex/

Categories : Mongodb

update nth document in a nested array document in mongodb
Q1: If you update with permalink 'jaiho' instead of 'haha', it most certainly updates the email; > db.posts.update({"permalink" : "jaiho"},{$set:{"comments.0.email":1}}) > db.posts.find() ..., "email" : 1 },... Q2: Same goes for this include; > db.posts.update({"permalink" : "jaiho"},{$inc:{"comments.0.num_likes":1}}) > db.posts.find() ..., "num_likes" : 1 },...

Categories : Arrays

MongoDB: If document exists VS If document does not exist
The question is how you determine that "the document" exists? Usually, you'd do this using a unique id. Now MongoDB's ObjectId comes to the rescue because it contains a timestamp already. This can also be used in queries. Instead of using an integer field User_ID, you might want to consider calling the field _id and use the ObjectId data type so you get that functionality for free. > db.test.insert({"foo" : "test"}); { "_id" : ObjectId("51cb763e58bb4077aea65b3d"), "foo" : "test" } > var foo = db.test.findOne(); > foo._id.getTimestamp(); ISODate("2013-06-26T23:16:14Z")

Categories : Mongodb

ElasticSearch and MongoDB
no you need not run that every time a collection gets updated. a index is refreshed according to the refresh interval in config or manually by invoking "_refresh" yes there is support for paging,sorting using fields "from", "size", "sort" in the query please see ElasticSearch Pagination & Sorting yes you can search in anyfield please see http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query/

Categories : Javascript

Relational Database Inheritance foreign keys and primary keys
It would probably make sense to have the child class tables reference the parent class, instead of the other way around. Then you can have an id column on the Entity table which is unique and foreign keys from the children to their parent instances. Presumably this will help when you want to use the data to instantiate an object in your code as well, since you should know which class you are instantiating and only care about its ancestors, not its children.

Categories : Database

Does ElasticSearch have the same _id functionality that mongodb does?
As far as I know Elasticsearch does not have that functionality. When you index a document without specifying the id. The id is automatically generated and it is a hash, so you can't guess the date from it :( . About the second question, you don't need a db hit to get the id because it is returned when you index the doc. curl -XPOST 'http://localhost:9200/test/test/' -d ' { "test_field": 1 }' {"ok":true,"_index":"test","_type":"test","_id":"T7KTroYsT86JlxCtclfScA","_version":1}

Categories : Mongodb

How to make elasticsearch document ttl work?
Here is what works for me: curl -XPUT 'http://localhost:9200/blog/user/_mapping' -d '{"user": {"_ttl": {"enabled": true, "default": 5000}}}' curl -XPUT 'http://localhost:9200/blog/user/phb' -d '{"name" : "Pointy-Haired Boss"}' sleep 60 # this is the default deletion interval for the expired documents curl -XGET http://localhost:9200/blog/user/phb # 404

Categories : Elasticsearch

Elasticsearch get position of phrase in document
Will simple highlighting suffice? Even if not, a brute-force solution would be to set the highlighting pre_tags to a programmably-identifiable value and calculate the offset from that. Speed it up by setting term_vector to with_positions_offsets in the mapping to use lucene's fast-vector-highlighter: { "type_name" : { "content" : {"term_vector" : "with_positions_offsets"} } } If that's not acceptable, check out this answer for information on how the offsets are stored internally. EDIT: Based on your edit, I'm not sure how having the offset would help that much. I mean, unless you're displaying preformatted text or some other fixed layout, how would you know where on the rendered page the offset corresponds? I think the most elegant solution is to use pre_tags and post_

Categories : Elasticsearch

How to get the best children for each parent document in Elasticsearch?
Have you found a way to do this? If not, once v1.4.0 of ElasticSearch will be released, you should be able by combining two aggregations : A children aggregation (will be included in 1.4.0) based on the parent type A A top_hits aggregation (added in 1.3.0) with size 1 to retrieve only the best matching child.

Categories : Elasticsearch

Index only some fields in ElasticSearch from mongodb
I never used river plugins, but one thing that I know is you can control index upon fields through mapping or template. For each field, you can specify the property "index", in mapping or template, to three different options: analyzed, not_analyzed, no. This is the official documentation. Set to analyzed for the field to be indexed and searchable after being broken down into token using an analyzer. not_analyzed means that its still searchable, but does not go through any analysis process or broken down into tokens. no means that it won’t be searchable at all (as an individual field; it may still be included in _all). Setting to no disables include_in_all. Defaults to analyzed. If you want your field still be searchable, go with "not_analyzed", otherwise "no". The type of fields sh

Categories : Mongodb

Update ElasticSearch Document while maintaining its external version the same?
The time for which elasticsearch keeps information about deleted documents is controlled by index.gc_deletes parameter. By default this time is 1m. So, theoretically, you can decrease this time to 0s, wait for a second, delete the document, index a new document with the same version, and set index.gc_deletes back to 1m. But at the moment that would work only on master due to a bug. If you are using older version of elasticsearch, you will not be able to change index.gc_deletes without closing the index first. There is a good blog post on elasticsearch.org web site that describes how versions are handled by elasticsearch in details.

Categories : Elasticsearch

fancy/efficient way of checking if a document exists at elasticsearch
I would just use the get api which returns a 404 if the object doesn't exist, otherwise the object itself. If you use the Java API you'll find an isExists method in the GetResponse object. If the _id field you are referring to is not included in your documents, saying fields=_id wouldn't give you back either the _source nor any specific field under fields. But you would get back the _id in the header of the response anyway. If you are using the REST api you can use the following: curl -XHEAD 'http://localhost:9200/twitter/tweet/1 it won't return the document back but just 404 if not found, 200 otherwise. The body of the response will also contain the exists flag too, with the same meaning. What's interesting is that using the HEAD method maps to a get request internally, that's why i

Categories : Elasticsearch

Elasticsearch mongodb river tailable curosr issue
elasticsearch mongodb river [1] uses the recommended way [2] to query tailable cursor in MongoDB If you observe extra query in oplog.rs I'd suggest to try contacting MongoDB and especially MongoDB Java Driver [3]. I am happy to fix any issue in elasticsearch mongodb river. [1] - https://github.com/richardwilly98/elasticsearch-river-mongodb/blob/master/src/main/java/org/elasticsearch/river/mongodb/MongoDBRiver.java#L1326 [2] - https://github.com/mongodb/mongo-java-driver/blob/master/examples/ReadOplog.java#L47 [3] - https://github.com/mongodb/mongo-java-driver

Categories : Elasticsearch

Accessing document string values in an ElasticSearch native script
You compiled your code with an older version of elasticsearch (0.20.x or older) than the one you're using at runtime (0.90.x). The error you get says that the not found method has a StringArrayRef return type. It seems weird since you assign the result to a List<String>, but that works at compile time since the StringArrayRef class extended AbstractList<String> before it was removed from the elasticsearch codebase. I guess the method was removed during the fielddata refactoring that has been done a few months ago. The method that you see on master and 0.90.x returns now a List<String>. Just update your elasticsearch dependency (use the same that you're using at runtime), recompile and you should be good to go!

Categories : Java

Handle document processing that exceed memory limits with Elasticsearch
Your question seems to be "How do I serialize 100,000 ElasticSearch JSON objects into Ruby objects without running out of memory?". A better question would be: "How do I run calculations on 100,000 ElasticSearch documents as easily and efficiently as possible?". Since we don't know what kind of calculations you are trying to run, we'll have to keep the answer general. Take neil-slater's suggestion and do as much in ElasticSearch as possible. For instance, ES has lots of nice statistical calculations you can do in the DB/store. Do preprocessing on insertion of new documents. For instance, if you know you are going to want to get counts, averages, or some other calculation against the entire set, just calculate the stats for each item before storing it in ES. If you are using Tire in Rails

Categories : Ruby

Mongodb: Store a tree as one nested document or store one document per node?
It looks like you are going to lots of operations in different levels of nested nodes in the tree. Although MongoDB can store a structure like you describe, it is not very good at allowing you to do update at lots of nested levels. Therefore I would recommend you to store each node as it's own document, and look at where you store the parent-child relations. Remember to optimise the schema for data operations. I'd go with your "way 1" in this case. If you would not have to change the tree a lot, and you have say 1000x more read than write operations to the tree, then you could consider using "way 2" and just deal with the extra work it takes to update the nodes at a few levels deep.

Categories : Json

Ordering by several keys in MongoDB
The syntax you use only works if your JavaScript implementation preserves the order of keys in objects. Most do, most of the time, but it's up to you to be sure. For the Nodejs official driver, there is an alternate form that works with an Array: db.collection.find().sort([[age, -1], [score, 1]]); Drivers for other languages might have something similar.

Categories : Javascript

Removing the duplicate keys of a JSON array and getting its JSON path
Modify the function getKeys to this : function getKeys(keys, obj, path) { for(key in obj) { var currpath = path+'.'+key; keys.push([key, currpath]); if(typeof(obj[key]) === 'object') { if(obj[key] instanceof Array){ //added check for array keys.push(['0', currpath+'.0']); getKeys(keys, obj[key][0], currpath+'.0'); } else getKeys(keys, obj[key], currpath); } } } Update function getKeys(keys, obj, path) { for(key in obj) { var currpath = path+'.'+key; addKeys(key, currpath); //keys.push([key, currpath]); if(typeof(obj[key]) === 'object') { getKeys(keys, obj[key], currpath); } } } function addKeys(key, currpath) { //check before adding present = keys.map(func

Categories : Javascript

MongoDB Aggregation Multiple Keys
mapReduce can be used to solve the problem. 1) define the following map function var mapFunction = function() { var key = this.department; var nb_match_bar2 = 0; var status_exist = 0; var status_absent = 0; if( this.status=="exist" ){ status_exist = 1; }else{ status_absent= 1; } var value = { department: this.department, statuses:{ exist: status_exist, absent: status_absent } }; emit( key, value ); }; 2) define the reduce function var reduceFunction = function(key, values) { var reducedObject = { department: key, statuses: { exist: 0, absent:0 } }; values.forEach( function(value) { reducedObject.statuses.exist += value.statuses.exist; reducedObject.statuses.absent += value.statuses.absent; }

Categories : Mongodb

how do hashed sharding keys work in mongodb?
A hashed key uses the data in the document, and a standard hash function (MD5) to generate a hash value for the document. For example, suppose your index is on {name: 'hashed'} and you have these documents: {name: 'John', height: 73} {name: 'Zardosht', height: 68} MongoDB will run 'John' and 'Zardosht' through the MD5 hash function and get random-looking byte sequences. Then, it'll use those byte sequences as the actual B-tree key to use in the index. When you want to do a query for {name: 'John'}, it does the MD5 hash again, gets the same byte sequence it would have gotten before, and then actually does an index lookup for that value. You can play with the mongo shell and actually see what these values will be for yourself: % mongodb-linux-x86_64-2.2.5/bin/mongo

Categories : Mongodb

Get the first document using $in with mongodb
You cannot add a limit to $in but you could cheat by using the aggregation framework: db.collection.aggregate([ {$match:{serial:{$in:[list_of_serials]}}}, {$sort:{_id:-1}}, {$group:{_id:'$serial',type:{$first:'$type'},sensors:{$first:'$sensors'},id:{$first:'$_id'}}} ]); Would get a list of all first found of each type. Edit The update will get the last inserted according to the _id.

Categories : Mongodb

Does indexes on dynamic nested keys works in mongodb?
In your original schema, each embedded object creates a unique field path in the document, so we can't effectively index those. A good illustration of these index limitations (and an example of refactoring one particular case) can be found in this presentation. In your case, you might benefit from a taxonomy schema, akin to this question, which I answered earlier today. MongoDB Content Taxonomy Schema presents a good suggestion for a schema requiring arbitrary key/value pairs associated with a record. Additionally, it describes creating a compound, multi-key index (on two embedded fields within the same array) and using $elemMatch to query by a key/value pair. This example entails an array of objects, each of which has a key and value field. An alternative schema could be an array of str

Categories : Mongodb

MongoDB: Sort distinct keys by number of occurances
One possible solution is a simple map / reduce. The advantage is that you could use it to aggregate additional information, such as the number of commits, committers, files... It might be too heavy a solution for what you want, though. I'm not entirely familiar with the modern aggregation framework, but I believe that if there's a solution other than map / reduce, that's where you're likely to find it.

Categories : Mongodb

Add a value to an array in a MongoDB document
You're looking for the $push operator. $arrayValueToAdd = array ( '22' => 1 ); $collection->update( array('email' => $user['email']), array( '$push' => array( 'emailQueries' => $arrayValueToAdd ) ) );

Categories : PHP

How to sum every fields in a sub document of MongoDB?
You have made the classical mistake to have arbitrary field names. MongoDB is "schema-free", but it doesn't mean you don't need to think about your schema. Key names should be descriptive, and in your case, f.e. "S2" does not really mean anything. In order to do most kinds of queries and operations, you will need to redesign you schema to store your data like this: _id:... Segment:[ { field: "S1", value: 1 }, { field: "S2", value: 5 }, { field: "Sn", value: 10 }, ] You can then run your query like: db.collection.aggregate( [ { $unwind: "$Segment" }, { $group: { _id: '$_id', sum: { $sum: '$Segment.value' } } } ] ); Which then results into something like this (with the only document from your question): { "result" : [ {

Categories : Mongodb

mongodb - one document in many collections
Try looking at the problem inside-out: Instead of having items inside categories, have the items list the categories they belong into. You'll be able to easily find all items that belong to a category (or even multiple categories), and there is no duplication nor any need to keep many instances of the same item updated. This can be very fast and efficient, especially if you index the list of categories. Check out multikey indexes.

Categories : Database

query the value of a sub document in mongodb
You can use dot notation in the projection parameter of find to do this. In the shell: db.test.find( {_id : ObjectId("5207fe359b88bfa6f90a82b0")}, {'members.5207fe359b88bfa6f90a82af': 1, _id: 0}) Returns: { "members": { "5207fe359b88bfa6f90a82af": [ "Admin", "User" ] } }

Categories : Mongodb

Mongodb error : Can't extract geo keys from object, malformed geometry?
The polygon object in geoJSON requires first point ([lon, lat]) to be same as last point. By making this change: {type: "Polygon", coordinates: [ [ [ 103.8324334524412, 1.284232321447769 ], [ 103.8342325475588, 1.284232321447769 ], [ 103.8342325469261, 1.282433678236006 ], [ 103.8324334530738, 1.282433678236006 ], [ 103.8324334524412, 1.284232321447769 ] ] ]} The insert query works fine.

Categories : Mongodb

Mongodb doctrine "zero-length keys are not allowed, did you use $ with double quotes"
if someone interest, i found solution by using separate "or" expressions: $q = $this->createQueryBuilder(); $keywords = explode(" ", $keyword); foreach ($keywords as $keyword) { $r = new MongoRegex('/'.$keyword.'/i'); $q->addOr($q->expr()->field('firstName')->equals($r)); $q->addOr($q->expr()->field('lastName')->equals($r)); } return $q->getQuery()->execute(); hope my experience will help

Categories : PHP

stringified json vs json object memory usage in mongodb
You can check the size of document with Object.bsonsize() in the mongoshell For example: > Object.bsonsize({}) 5 > Object.bsonsize({a:''}) 13 > Object.bsonsize({a:'',b:''}) 21 > Object.bsonsize({a:'',b:'',c:''}) 29 > Object.bsonsize({a:{a:'',b:'',c:''}}) 37 > Object.bsonsize({a:{a:'',b:''}}) 29 > Object.bsonsize({a:{a:''}}) 21 > Object.bsonsize({a:"{a:'',b:'',c:''}"}) 29 > Object.bsonsize({a:"{a:'',b:''}"}) 24 > Object.bsonsize({a:"{a:''}"}) 19 > Object.bsonsize({a:""}) 13 > Object.bsonsize({ab:""}) 14 > Object.bsonsize({abc:""}) 15 It looks like for me that empty doc is 5 byte one empty string is 7 byte plus each character in the name is 1 byte and each in the content is 1 byte.

Categories : Json

mongodb update with many embedd document
When you call Save, base on your [BsonId] field driver decides what to do with this document: if document with Id does not exist in collection, it will insert this document if document with Id exists in collection, it will update this document (based on your Id). Update means it will replace all fields with the new fields from your entity Here is an example how save method works in pseudocode (the same as in C# driver): function save( doc ) { if( doc["_id"] ) { update( {_id: doc["_id"] }, doc, { upsert: true } ); } else { insert(doc); } }

Categories : Mongodb

how to update a mongodb document from node.js?
Change this line: ({_id:doc._id},$set:{scores:zz}); To: ({_id:doc._id}, { $set:{scores:zz}} ); This should also probably be wrapped with a callback, to catch errors: db.schools.update({_id:doc._id}, {$set:{scores:zz}}, function(err, result) { if (err) //do something. });

Categories : Node Js

delete element from document in MongoDB using C#
After searching on the web for a while, i found it myself. For those who want to know my solution: for (int j = 0 ; j < idlist.Count ; j++) { var queryDeleteValue = new QueryDocument("_id", idlist[j]); var update = Update.Unset("Flash_point"); collectionInput.Update(queryVerwijderValue, update); } So, first of all, i have a query-variable that selects the right document. Then i make an update variable where i unset the element 'Flash_point'. The last step is to do the actual update (with the parameters 'queryDeleteValue' and 'update').

Categories : C#

Single qoute in MongoDB document
I had same problem before 2 days and i have solved it using below query. below is the solution for me. SELECT * FROM tags WHERE `name` = 'limit\''s' LIMIT 0 , 30 SEE HERE

Categories : PHP



© Copyright 2017 w3hello.com Publishing Limited. All rights reserved.