w3hello.com logo
Home PHP C# C++ Android Java Javascript Python IOS SQL HTML videos Categories
   » Home » Dmoz data
Parsing DMOZ dumps for category queries in Python
I started with https://github.com/kremso/dmoz-parser and made a simple topic filter : https://github.com/lawrencecreates/dmoz-parser/blob/master/sample.py#L6 class LawrenceFilter: def __init__(self): self._file = open("seeds.txt", 'w') def page(self, page, content): if page != None and page != "": topic = content['topic'] if topic.find('United_States/Kansas/Localities/L/Lawrence') > 0 : self._file.write(page + " ") print "found page %s in topic %s" % (page , topic) def finish(self): self._file.close()

Categories : Python

scrapy newbie: tutorial. there is an error when running scrapy crawl dmoz
Can you please verify the name of the spider in the project you have created "demoz" or "dmoz" ? You specified "dmoz" as the spider name in the command d:Scrapy workspace utorial>scrapy crawl dmoz

Categories : Python

When calling a web service that returns data, is the data that's being returned over HTTP limited to binary data?
I believe what you are asking is, "can you return content types?" If my interpretation of your question is correct, then the answer is yes. This article might be helpful Or did I completely bork the interpretation?

Categories : Web Services

Creating stored procedure with 2 different sets of data (using value from first data as parameter for 2nd data)
select t1.[start], t1.[end], count(*) from table1 as t1 left outer join table2 as t2 on t2.timestamp between t1.[start] and t1.[end] group by t1.[start], t1.[end] sql fiddle demo

Categories : Sql Server

how to store data in NS-Mutable-data in did-receive-data and than write to file
It's better to use asynchronous requests for the task like this. You can use the same ASIHTTPRequest class, but with block approach. Try to write the code similar to this: -(void) verifyReceipt { NSURL *theURL = [NSURL URLWithString:@"http://testing.io/dev.php/video/verifyReceipt"]; NSMutableURLRequest *theRequest = [NSMutableURLRequest requestWithURL:theURL cachePolicy:NSURLRequestReloadIgnoringCacheData timeoutInterval:10.0f]; [theRequest setHTTPMethod:@"POST"]; NSString *param1 = [self getParam1]; // getParam1 - some method to get useful data for request's body NSNumber *param2 = [self getParam2]; NSString *postString = [NSString stringWithFormat:@"param1=%@&param2=%@", param1, param2]; [theRequest setHTTPBody:[postString dataUsingEncoding:NSUTF8StringEnco

Categories : Iphone

Save external data in directory Android/data or data/
you should rely on the API and you should avoid to hardcode path. File externalDir = Context.getExternalFilesDir(null); that path is returned by Context.getExternalFilesDir, which returns your secondth path

Categories : Android

Which data type is more apt to store generic data without any data loss
From my experience with .Net and Excel, it appears that it is good to have SQL Variant in the database. If necessary, either in SQL or in an application casting can be made (if you know the type of data) . In SQL you can use for this purpose SQL_VARIANT_PROPERTY (expression, property) with gives you the type of stored data.

Categories : C#

data.table operations on subsetted data and plotting data
This will cycle through an calculated all appropriate ratios for(kk in unique(DT[['analyte']])){ DT[, (paste('ratio',kk,sep='.')) := mass / DT[force(kk)][['mass']]] } # create the plot DT['b', plot(x=time,y= ratio.e,type='n')] # add the lines (creating a colour vector `cc` first` cc <- c('red','green','blue') DT['b', lines(x = time,y = ratio.e, col = cc[src]), by = src] Or, using ggplot2 library(ggplot2) ggplot(DT['b'], aes(x = time,y=ratio.e, colour = factor(src))) + geom_line()

Categories : R

how to fetch the data from parent table data based on child table data in many to many relationship in bidirectional using hibernate and spring
${course.followedBy} should return the list of student under a current course. Then you can itterate it ... <c:forEach items="${coursesList}" var="course"> ... <c:forEach items="${course.followedBy}" var="student"> </c:forEach> </c:forEach> ...

Categories : Spring

Using data attributes and jQuery how can I select elements by data attribute name AND data attribute value
you can use attribute selector var y = $('.my-class[data-id="' + i + '"]'); since the selector .my-class is repeated, you can cache it var els = $('.my-class'); for(var i=0; i<x.length; i++){ var y = els.filter('[data-id="' + i + '"]'); //trying to select here y.html('input' + i); } Demo: Fiddle

Categories : Javascript

when applying the BMA packages in R : Error in terms.formula(formula, special, data = data) : '.' in formula and no 'data' argument
Here is your answer: test.bic.surv <- bic.surv( x[, 3:ncol(x)], x$crisis1, x$cen1, factor.type=FALSE, strict=FALSE, nbest=2000, maxCol=50 ) You have to provide maxCol parameter. Default is 30 so it is probably not enough for your needs.

Categories : R

Add a data to a data frame from another smaller data frame based a condition
Calling your first dataset 'bigdata', and second 'flowobs', this works as with apply: bigdata$FlowObs <- apply(bigdata, 1, function(d) { flowobs[d[["Event"]]>flowobs$Start & d[["Event"]] <= flowobs$Finish,]$FlowObs }) > bigdata Event FlowRate FlowObs 1 2013-05-25 17:29:31 0.3739769 0.3307309 2 2013-05-25 17:37:31 0.5208873 0.3307309 3 2013-05-25 17:39:01 0.4235871 0.3307309 20 2013-05-26 01:16:31 0.3010403 0.3307309 21 2013-05-26 01:38:41 0.3054283 0.3307309 22 2013-05-26 02:01:01 0.3919175 0.3307309 116 2013-05-28 10:58:11 0.3851580 0.3286909 117 2013-05-28 11:11:12 0.3981671 0.3286909 118 2013-05-28 11:16:21 0.4075771 0.3286909 253 2013-05-31 08:31:11 0.3543576 0.3277443 254 2013-05-31 08:53:21 0.3553817 0.3277443

Categories : R

How does one summarise data grouped by a column and date, accounting for dates with missing data
WITH DateTable AS ( SELECT CONVERT(date, DateAdd(WEEK, -1, GETDATE())) AS [DATE] UNION ALL SELECT DATEADD(dd, 1, [DATE]) FROM DateTable WHERE DATEADD(dd, 1, [DATE]) < CONVERT(date, GETDATE()) ) SELECT Tags.Name, dt.DATE as Date,COUNT(Events.ID) as Count FROM DateTable dt CROSS JOIN Tags Tags LEFT JOIN Events ON dt.[DATE] = CONVERT(date, Events.[CreatedAt]) and Tags.Id = Events.TagId where tags.Name like '%sample%' GROUP BY Tags.Name, Date ORDER BY Tags.Name, Date; fiddle

Categories : SQL

Use index of a list of data.frames to apply a function in certain elements of a data frame
Leveraging data.table here is helpful since you can easily subset your data using .( ) syntax and extremely fast, especially with large data compared to working with, say subset library(data.table) # convert to data.table, setting the key to the column `A` DT <- data.table(df, key="A") p.col.names <- paste0("P", 1:6) results <- lapply(list.1, function(ll) DT[.(ll)][, lapply(.SD, Fisher.test), .SDcols=p.col.names] ) results side note You might want to fix the names of list.1 so that the results form lapply are properly named # fix the names, helpful for the lapply names(list.1) <- lapply(list.1, names) results: $AA P1 P2 P3 P4 P5 P6 1: 0.04770305 0.1624142 0.2899578 0.029753 0.1070376 0.17549 $BB P1

Categories : R

How to sort data from a text file in java and put the data into an array to create an object?
You need to make the below changes in your for loop. for (int i = 0; i < orders.length; i++) { String data = scan.nextLine(); // you need to use nextLine to read a whole line String[] val = data.split(" "); String type = val[0]; // Since its a String double price = Double.parseDouble(val[1]); String location = val[2]; // Since its a String double taxRate = 0.0; // Default values double tariff = 0.0; // Default values try { // Incase they are not present - error handling taxRate = Double.parseDouble(val[3]); tariff = Double.parseDouble(val[4]); } catch (ArrayIndexOutOfBoundsException e) { } orders[i] = new Order(type, price, location, taxRate, tariff); }

Categories : Java

F# Immutable data structures for high frequency real-time streaming data
Suppose your dataObj contains a unique ID field, then any set data structure would be fine for your job. The immutable data structures are primarily used for functional style code or persistency. If you don't need these two, you can use HashSet<T> or SortedSet<T> in the .Net collection library. Some stream specific optimization may be useful, e.g., keeping a fixed-size Queue<T>for the most recent data objects in the stream and store older objects in the more heavy weight set. I would suggest a benchmarking before switching to such hybrid data structure solutions. Edit: After reading your requirements more carefully, I found that what you want is a queue with user-accessible indexing or backward enumerator. Under this data structure, your feature extraction operation

Categories : Database

Why does my Datagram socket stop recieving data, even though it recieved data perfectly fine?
It looks like your Server class is designed to be used as the Runnable of some worker thread. I suspect that the problem is that the run() method is dying due to an uncaught exception, and that the worker thread is simply terminating. I recommend that you configure a default uncaught exception handler to (at least) log the stack trace for the fatal exception. Refer to the Thread javadocs for details.

Categories : Java

Repeated querys to a db with JDBC or dump data to ArrayList and iterate over list whenever I want some data?
You have different options based on how big your tables and how scalable you need your system to be. If you have a pattern in showing the daily appointments, then I suggest you extract that pattern to have a generic solution to show the appointments of a day. On top of this if you have validations to be running for a selected day, keep it simple and safe in your database, that way the whole point of persistence is not lost also. Have a look at in memory databases as well.

Categories : Java

Merging multiple, unaligned data-frames into single pandas data-frame
If you have the data-frames in a list (the actual data-frame objects, I mean, not their names) that looks something like this: dflist = [df1, df2, df3, df4, df5, df6, df7, df8] then the following code should do what it seems you're looking for: frame = {} for idx, df in enumerate(dflist): n = idx+1 # Since lists are 0-indexed name = "df{0:d}.Close".format(n) close = df["Close"] frame[name] = close You could do this more compactly with a dict comprehension, but in example code I prefer to spell things out more explicitly for ease of understanding. For reference, the dict comprehension would look something like this: {"df{0:d}.Close".format(idx+1): df["Close"] for idx, df in enumerate(dflist)}

Categories : Python

Whats the best way to convert array of hashes data to specific graph format data
Because you are not printing your value. The console is only displaying it, double quotes are escaped. Otherwise how would you know where my string: "my "string" what" ends? If you just do puts @specific_details.to_s you will not see those double quotes escaped.

Categories : Ruby

How to add a data in a column if that data doesnt remain in a table for a specific foreign key in sqlite
You're basically not realising that this is just a normalization issue. This becomes evident when you try to insert "math" twice in the same table. This is a many-to-many and not a one-to-many relationship. Rule of thumb: Many-to-many relationships always generate a table. The appropriate table structure would look like this (sample data included): Categories: (cat_id) cat_id cat_name 1 Science 2 Arts Items: (item_id) item_id item_name 1 math 2 literature 3 physics Categories_Items (cat_id, item_id) cat_id item_id 1 1 1 3 2 2 In round brackets you'll see the primary keys. Bear in mind that the last table has a compound PK. That will allow you not to repeat math for the sam

Categories : Mysql

Encrypt data using c# and decrypt it using openssl api, why there are lots of trashy padding at the end of decrypted data?
The issue is that you should use the decrypted size instead of directly treating the output of the decryption as a null terminated string. RSA_private_decrypt() returns the size of the recovered plaintext.

Categories : C#

How can I view contents in Sql Data Tools Visual Studio 2010?
You can add calculated field with code CONVERT(VARCHAR(35), YourField, 2) Try: DECLARE @bin VARBINARY(MAX) = 0x5BAA61E4C9B93F3F0682250B6CF8331B7EE68FD8 SELECT CONVERT(VARCHAR(35), @bin, 2)

Categories : SQL

How to access model data from a unit/integration test with ember-data 1.0 beta
For tests you can always lookup the store in the application container by doing: App.__container__.lookup('store:main'); So, something like this should work: test('user is authenticated', function() { var store = App.__container__.lookup('store:main'); var me; me = store.find('me'); me.set('IsAuthenticated', true); ok(me.get('IsAuthenticated'), "user should be authenticated."); }); Hope it helps.

Categories : Unit Testing

objective-c passing data from a delegate to the initial calling class (getting data for a uitableview)
Try this , hope helps u .. :) #import <Foundation/Foundation.h> #import "AppDelegate.h" @class FbApi; @protocol FbApiDelegate <NSObject> -(void)didLoadRequestForTableview:(NSMutableDictionary *)aDict;//change the delegate method @end @interface FbApi : NSObject @property (assign, nonatomic) id <FbApiDelegate> delegate; @property (strong, nonatomic) NSMutableDictionary *json; -(void)fbIDInformation:(NSString *)fbID to:(NSMutableDictionary *)dict; @end // FbApi.m #import "FbApi.h" @implementation FbApi -(id)init { self = [super init]; if (self) { } return self; } -(void)fbIDInformation:(NSString *)fbID to:(NSMutableDictionary *)dict { NSLog(@"i am in FbApi mode. DOING A NEW REQUEST"); NSString *fbquery = [NSStrin

Categories : Objective C

multipart/form-data File upload request from Flash to GAE misses data
Are you receiving the POST request but missing all the POST data? It could be due to the HTTP headers were not sent properly in Flash. Try to read and parse directly from the HTTP request body and you should get something.

Categories : Python

Genymotion. How to pull database from device. my /data/data/ folder appears to be empty
I managed to kind off go around the issue so I post my solution in case someone will find it helpful. Previously I've been creating VM's with no google apis, superUser app crashed every time I wanted to root the phone. so I created VM that is provided with google play and downloaded SQL Debugger App. I also use this code in my test suite and so I fetch db to SD-card at the end of every test I run. That way I don't need to have superuser permissions. public static void copyDbToSd(Context context) { File dbFile = context.getDatabasePath((String) DatabaseHandler.DATABASE_NAME); InputStream myInput; try { myInput = new FileInputStream(dbFile); OutputStream myOutput = new FileOutputStream( Environment.getExternalStorageDirectory()

Categories : Android

should reassigning my viewmodel clear ui data in
It is a little hard to tell without knowing what your presenter is doing, but in general you will want to avoid calling ko.applyBindings on content that has already been bound. If you are trying to load a new propertyModel into your view, then it generally helps to use either the with or template binding against an observable like: <div data-bind="with: currentPropertyModel"> ...content here </div> Then, you define a currentPropertyModel as an observable, call applyBindings a single time, and then just set and clear currentPropertyModel with the model that you want rendered. When you clear it, the content will be removed and when you set it to a new one, then it will be rendered with that new model as the context.

Categories : Knockout Js

Moving javascript data series to html data attribute breaks Highchart
Try passing the data to the client in a content_tag in the view and rename @data to be @my_data: <%= content_tag "div", id: "chart_data", data: {my_data: @my_data} do %> <% end %> Reload the page, open the JavaScript console, and use jQuery to see if the JavaScript objects are formatted properly: $('#chart_data').data('my_data') In a realistic example, you will want to pass a collection of Ruby objects to the client and this technique is suitable for converting an array of Ruby objects to an array of JS objects. There is a good RailsCast on passing data to JS.

Categories : Ruby On Rails

Appending data to properties file, comments disappear & order of data changed
It is not straight forward to preserve the comments of properties file. There are no methods on java.util.Properties to handle comments. Comments are simply ignored when reading the file. As only the key value pairs are loaded when we do properties.load and hence when you save it back the comments are lost. Check the link below, there is one solution to achieve what you need but not the elegant way: http://www.dreamincode.net/forums/topic/53734-java-code-to-modify-properties-file-and-preserve-comments/

Categories : Java

How to structure data in a relational schema where related data is not a case of a simple join
As was suggested by Gilbert Le Blanc I think the solution here is to link missions to teams, even if the team has a single person in it. This removes the possibility that someone assigns a person AND and team to a mission and removes the need to have a UNION query. This will however force you to do 3 joins (Missions -> Teams, Teams -> IndividualsInTeams, IndividualsInTeams -> Individuals). If that is too costly then I suggest you create a static view on your data everyday if you can tolerate data that is not live. Depending on your database size the view may or may not be costly to generate. Depending on the time cost you may want to update it twice a day to have more accurate data. That would save you time for your most important queries (what missions are assigned to user X ?).

Categories : Database

Vectorized meta data computation based on multiple columns on R data.frame
aggregate is your friend for these tasks: Example data: set.seed(23) test <- data.frame( obs=sample(LETTERS[1:2],20,replace=TRUE), len=sample(c(10,20),20,replace=TRUE), src=sample(LETTERS[24:25],20,replace=TRUE) ) Aggregate it: aggregate(obs ~ src + len,data=test, function(x) prop.table(table(x))) src len obs.A obs.B 1 X 10 0.6000000 0.4000000 2 Y 10 0.2000000 0.8000000 3 X 20 0.2500000 0.7500000 4 Y 20 0.1666667 0.8333333

Categories : R

using java script to select Google chart data to change the data's series
What you want to do, is to set all zeros equal to nulls, and set the interpolateNulls:true parameter in your options when you draw your chart: // Set all zeros in our dataTable to null. var rows = data.getNumberOfRows(); var cols = data.getNumberOfColumns(); for (var i = 0; i < rows; ++i) { for (var j = 0; j < cols; ++j) { if (data.getValue(i, j) == 0) { data.setCell(i, j, null); } } } // etc.. var element = document.getElementById('visualization'); var chart = new google.visualization.LineChart(element); chart.draw(data, { interpolateNulls: true });

Categories : Javascript

Is it possible to restrict the amount of data ('gate' the data) sent to unauthenticated (angularFire) clients in firebase?
You'll need to restructure your data in order to achieve this with Firebase. See this blog post for some background on this: https://www.firebase.com/blog/2013-04-12-denormalizing-is-normal.html One way to structure the data would be to create two separate lists, one for active items and another for inactive items. When an item becomes active or inactive, be sure to remove it from one list and put it in another. { "active": { 0: { "data": {...}, "sign": {...} } }, "inactive": { 0: { "data": {...}, "sign": {...} } } } In your security rules, you can simply prevent access to inactive items to unauthenticated users: { "rules": { "active": { ".read": true }, "inactive": { ".read": "auth != null" } } } Hope t

Categories : Javascript

Is there a way to make the form stop loading data from file after specific data has been found
you can read all the lines of the file, and exit when you find the info you need. Something like this might work for you: StreamReader reader = new StreamReader("YOUR PATH GOES IN HERE"); while(reader.Peek()>=0) { string yourString = reader.ReadLine(); if(yourString == "WHATHEVER YOU NEED") { //YOUR CODE HERE break; } } reader.Dispose(); reader.Close(); I hope that helps you!

Categories : C#

error at Data Flow Task the data type DT_WSRT and DT_I4 are incompatibale
In Flat file source you can specify what is the type of read data - probably it is still set to string(50) and that's why you can't compare string vs integer value. Another easy way out is to cast your string variable to integer in Condition: (DT_I4)Salary > 20000

Categories : Sql Server

Unable to see data in a string field in apache solr 3.6 when importing data from mysql
Try to remove the required= "false" attribute in your field (it should be the default anyway). Try to change the definition of your field with this: <field name="company_logo" type="string" indexed = "true" stored="true" multivalued="false"/>

Categories : Solr

Loss of data warning when copying to a data structure of smaller type
You can put #pragma to suppress the specific warning. > #pragma warning(push) > #pragma warning(disable: 4244) // possible loss of data > #include <algorithm> > #pragma warning(pop) This is of course compiler specific.

Categories : C++

Callback after Rails link with data-remote="true" and data-method="PUT"
With remote: true, Rails makes an ajax call to your controller action with format js. So, in your controller's action when you're ready to render the output you just need to specify format.js option as follows: def action ... respond_to do |format| format.html {...} # You probably have something like this already format.js {} # Add this line end Then, you need add a action.js.erb file in your app/views/app/ directory and execute the javascript code that you wanted to execute on onclick there.

Categories : Ruby On Rails

How to insert association/child data into Ext.data.list as separate entries
Yes, create another store for locations. You may have tried a template like this: itemTpl: [ '{name}', '<tpl for="locations">', ', {address}', '</tpl>' ] But that will indeed only display the information, it won't let you interact with each location as an individual list item. You could get it working by hacking the view's doRefresh method, but that's just going against the lib's intention and other developer' expectations. Maybe the problem is that you can get the data only in this format, that is with locations as children of companies, and you can't get the server to send you a flat list of companies. In that case, I think the most meaningful approach would be to customize a reader to flatten locations from companies, and feed a standalone location

Categories : Extjs



© Copyright 2017 w3hello.com Publishing Limited. All rights reserved.