w3hello.com logo
Home PHP C# C++ Android Java Javascript Python IOS SQL HTML videos Categories
In Hadoop is it possible to specify the record delimiter for TextOutputFormat
Not without writing your own implementation of TextOuputFormat. TextOutputFormat uses LineRecordWriter to write records. This writer has the record separator hardcoded to . static { try { newline = " ".getBytes(utf8); } catch (UnsupportedEncodingException uee) { throw new IllegalArgumentException("can't find " + utf8 + " encoding"); } } and no option to change it... public synchronized void write(K key, V value) throws IOException { boolean nullKey = key == null || key instanceof NullWritable; boolean nullValue = value == null || value instanceof NullWritable; if (nullKey && nullValue) { return; } if (!nullKey) { writeObject(key); } if (!(nullKey || nullValue)) { out.write(keyValueSeparator); } if (!nullValue) { writeObjec

Categories : Java

Is it possible to create a 32bit standalone executable on a 64bit machine?
Probably the simplest way to do this is to install the 32 bit version of MATLAB and use the mcc from that installation. Don't worry about the warnings that you get about installing 32 bit MATLAB on 64 bit Windows. It works just fine. I have lots of different versions of MATLAB installed on my machine for development purposes, both 32 and 64 bit, all happily co-existing side by side. The documentation also mentions a -win32 option that can be passed to mcc: -win32 Run in 32-Bit Mode Use this option to build a 32-bit application on a 64-bit system only when the following are both true: You use the same MATLAB installation root (matlabroot) for both 32-bit and 64-bit versions of MATLAB. You are running from a Windows command line (not a MATLAB command line). Since

Categories : Matlab

Maintaining multiple standalone ruby scripts on same machine
How about using RVM or RbEnv? In this case, you will need to have each file in a separate folder with rvm/rbenv config.

Categories : Ruby

Upload file from my virtual machine to another virtual machine using hadoop hdfs
You might find webHDFS REST API useful. I have tried it to write content from my local FS to the HDFS and it works fine. But being REST based it should work fine from local FS of a remote machine as well, provided both machines are connected.

Categories : Hadoop

Hadoop one machine only run one task
I think you have to set the virtual machine options (this is for both map and reduce tasks): mapred.child.java.opts=-Xmx7000m If you have the new API supported you can specify it for the mapper only with: mapreduce.map.java.opts=-Xmx7000m I had similar problems and also logged the virtual machine heapsizes, more in this: small blog post about checking java heap sizes Note that also reducers are running on a node, so they might compete for memory, make sure to limit the number of reduce slots as well if necessary.

Categories : Java

DiskErrorException on slave machine - Hadoop multinode
Looks like you have hit this issue. Either apply the patch or download the fixed version, and you should be good to go. HTH

Categories : Hadoop

Where does map-reduce/hadoop come in in machine learning training?
Yes. There are many MapReduce implementations such as hadoop streaming and even some easy tools like Pig, which can be used for learning. In addition, there are distributed learning toolset built upon Map/Reduce such as vowpal wabbit (https://github.com/JohnLangford/vowpal_wabbit/wiki/Tutorial). The big idea of this kind of methods is to do training on small portion of data (split by HDFS) and then averaging the models and commutation with each nodes. So the model get updates directly from submodels built on part of the data.

Categories : Hadoop

How to access hadoop cluster (unix) from client windows machine
Probably it is not the best solution but I figure it out: 1) created "windows" user on linux machine 2) add that user to usergroup under which hadoop is running 3) hdfs-site.xml add prperty for dfs.permissions.supergroup to usergroup hadoop is running under That solved the problem for me. I do not claim that it is the best, definetevly not a propper user management.

Categories : Hadoop

Need help adding multiple DataNodes in pseudo-distributed mode (one machine), using Hadoop-0.18.0
Hadoop start/stop scripts use /tmp as a default directory for storing PIDs of already started daemons. In your situation, when you start second datanode, startup script finds /tmp/hadoop-someuser-datanode.pid file from the first datanode and assumes that the datanode daemon is already started. The plain solution is to set HADOOP_PID_DIR env variable to something else (but not /tmp). Also do not forget to update all network port numbers in conf2. The smart solution is start a second VM with hadoop environment and join them in a single cluster. It's the way hadoop is intended to use.

Categories : Hadoop

Counting delimiter in String and splitting the other character between the delimiter in SQL Server using loop
I think this will do the job. while @index1 < LEN(@symbol) begin set @index2 = CHARINDEX(',',@symbol,@index1) set @op = substring(@symbol,@index1,@index2-@index1) print @op set @index1 = @index2+1 end

Categories : Sql Server

Delimiter not working on MySQL 5.1
the problem is in the line INSERT INTO TD_PRINCIPIO_ACTIVO(`ID_TD_PRINCIPIO_ACTIVO`,`ID_TM_PRINCIPIO_ACTIVO`) VALUES(ID_PRODUCTO,LAST_INSERT_ID()); . You are trying to directly call a stored procedure LAST_INSERT_ID() in your second select statement. If LAST_INSERT_ID() returns a single value, try to save it in a variable. Then use that variable DECLARE last_id int default 0; SET last_id= SELECT LAST_INSERT_ID(); #if it is a function other wise use exec INSERT INTO TD_PRINCIPIO_ACTIVO(`ID_TD_PRINCIPIO_ACTIVO`,`ID_TM_PRINCIPIO_ACTIVO`) VALUES(ID_PRODUCTO,last_id);

Categories : Mysql

Which is better for running recommendations on a Hadoop cluster, Apache Mahout or using R with Hadoop (via hadoop streaming/RHIPE/RHadoop etc)?
Mahout integrates many algorithms, and it deals with three questions (1) Recommendation (Collaborative filtering algorithm) (2) Clustering (3) Classification R is just for statistic, so if you want to do these three tasks above, Mahout is a great tool. BTW, Mahout is a Maven project, so you need to install Maven.

Categories : R

Removing text between "|" delimiter and "," delimiter using shell script
Try sed: sed "s/|[^,]*//g" Result: h2co3-macbook:~ h2co3$ echo "name,title,email1|email2|email3,phone,address" | sed "s/|[^,]*//g" name,title,email1,phone,address h2co3-macbook:~ h2co3$

Categories : Regex

resultset.isLast() is not working standalone application
Edit: From the Oracle Docs: The method isLast() cannot always detect when the ResultSet is positioned on the last row and may return false instead of returning true when used on a FORWARD-ONLY ResultSet. Which is really wierd, as you defined your ResultSet not to be FORWARD-ONLY. : S

Categories : Java

php explode not working as expected with string as a delimiter
When your code echo end(explode('Item description', $body)); return everything or all $body as string that mean 'Item description' not found . And When return empty string that mean the last element of return array is empty string

Categories : PHP

Split at multiple delimiter without delimiter in the list
As arshajii points out, you don't need groups at all for this particular regexp. If you did need groups to express a more complex regexp, you could use noncapturing groups to split without capturing the delimiter. It's potentially useful for other situations but syntactically messy overkill here. (?:...) A non-capturing version of regular parentheses. Matches whatever regular expression is inside the parentheses, but the substring matched by the group cannot be retrieved after performing a match or referenced later in the pattern. http://docs.python.org/2/library/re.html So the unnecessarily complex but demonstrative example here would be: re.split(r'(?:[|])', data2)

Categories : Python

Hadoop error "exec: /usr/lib/hadoop-0.20/bin/hadoop: not found"
This means soft-link /usr/bin/hadoop is not pointing to right location check if hadoop command exist at /usr/lib/hadoop-0.20/bin/hadoop, and make /usr/bin/hadoop to point to right location that is the location of your hadoop installation directory

Categories : Java

sed not working on remote machine
Instead of getting in a quoting headache save your script into a file script.sh (use a sensible descriptive name) and run: $ ssh root@cnc-02 'bash -s' < script.sh

Categories : Linux

WIX CustomAction.CA.dll not working in Windows XP machine
You might be used .NET Framework 4.0 (CLR 4.0) in your custom action. It will not run in .NET Framework 4.0 or newer is not installed machine. In your XP machine .NET Framework 4.0 may not be installed. So that you’re custom action is not running. Use useLegacyV2RuntimeActivationPolicy property in your CustomAction.Config file and set all supported time version which you want to use (CLR 2.0 for .Net Framework 2.0 and 3.5, CLR 4.0 for .Net Framework 4.0 and 4.5)

Categories : Wix

Applet not working after updating JRE on my machine
Everything above this line is irrelevant: java.net.ConnectException: Connection refused: connect And this simply means that your applet is experiencing problems at the network level (not Java Security). The host to which it is connecting is refusing the connections, which usually means there is no service listening at that host:port.

Categories : Java

Working of Combiner and Reducer class in hadoop
The combiner is executed on each node after executing the map. Yet need to divide by 5 (the number of different courses) only at the very end of the summation. Therefore you only can divide at the end of the reducer and not at the end of the combiner. Basically you can either: Remove the combiner (but keep the reducer) Define a reducer that does exactly the same as the reducer but does not divide at the end

Categories : Java

Developer ID Code Signing working on one machine but not another
The Mac Book Air did not have extra Command Line Tools installed, which is what caused the problem. I needed to have Xcode 4.4 (or newer) and go to the Preferences Window > Downloads section and install the Command Line Tools from there for the code signing to work properly.

Categories : IOS

Decryption of XML file is working just on single machine
"gf154de4-12f1-430a-8210-63fde92fbb17" is not a RSA private key - it's way too short. You're assigning it to a field called KeyContainerName, so I would guess that it's just a name for a private key that's stored somewhere on your PC by the Windows encryption APIs. Since the other PC doesn't have that key installed, it doesn't work. I'm not a Windows encryption expert, so I don't know how to fix it, but I hope this helps you find out!

Categories : C#

document.write() not working on one machine, works on others
"The file contains no doctype declarations, so should be being parsed as standard HTML." Can you explicitly specify the doctype and test it? "Furthermore, the applet tag works perfectly fine if it's written in HTML. It only fails when it's written using document.write." to test whether the problem is with document.write() or document.write('') try to write something like document.write("hello world")

Categories : Javascript

r support vector machine e1071 training not working
Putting Roland's answer in the proper "answer" format: target is numeric sex is a factor Let me give a few more suggestions: it seems as if target really should be a factor. (It has only 2 levels, 0 & 1, and I suspect you're trying to classify into either 0 or 1.) So stick in a dataset$target <- factor(dataset$target) somewhere. right now, because target is a numeric, a regression model is being run instead of a classification. it's worthwhile to do a similar check for any of your variables before running a model (especially a model). In the case you gave, for instance, it's not obvious what col1 and col2 are. If either of them are a grouping or classification, you should also make them factors, too. In R, many functions have multiple ways in which they will run, depending

Categories : R

add the rules to open a port using iptable, then the hadoop cluster does not working
Most clusters i've ever worked with tend to be firewall less between the various hadoop nodes but operate on separate network with a gateway node between the normal network and the hadoop cluster (and it's this gateway node that has the firewall to control access to the cluster as a whole). If you do want to pursue running a firewall on the namenode, there are a couple of links i found with a google search "Hadoop iptables" that list a more complete set of iptable rules that your 1 liner. Obviously you'll need to amend for your IP network address space and hadoop port numbers, but they are a good starting point: Search on each page for "iptables": https://www.triumf.info/wiki/DAQwiki/index.php/HADOOP https://sites.google.com/a/lyz.jp/distributed-computing-research/subprojects/hadoop

Categories : Hadoop

Powershell Start-Process msiexec on a remote machine not working
You could try executing Start-Process with Passthru to see if an error is being returned: (Start-Process -FilePath msiexec.exe -ArgumentList $msiArgumentList -Wait -Passthru).ExitCode The other thing that may help is increasing your logging to /l*v Update 1 Can you try the following, just to check remote commands for msi are working, it should result in 1619. (Start-Process -FilePath msiexec.exe -ArgumentList "/i no.msi /quiet /qb!" -Wait -Passthru).ExitCode

Categories : Powershell

org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop/dfs/name is in an inconsistent state
This can be resolved by specifying your namenode dir to a different location is "hdfs-site.xml" in your Hadoop Configuration . Generally it takes default file://${hadoop.tmp.dir}/dfs/name .. So , after every reboot the /tmp directory is cleared and NameNode data is gone

Categories : Hadoop

how to add maven classpath when submitting hadoop task to a hadoop node
maybe you can build a soft links in $HADOOP_PREFIX/libexec/lib/ which point to your maven repository path. If you use eclipse,you can add the maven repository path to the project build path: window->preferences->java->Build Path->ClassPath Variable create a M2_REPO = your repository path

Categories : Maven

org.apache.hadoop.conf.Configuration does not exist in hadoop-core.jar
Add hadoop-common to your pom.xml <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.0.0-cdh4.3.1</version> </dependency>

Categories : Maven

How to copy only file permissions and user:group from one machine and apply them on another machine in linux?
How about this? #!/bin/bash user="user" host="remote_host" while read file do permission=$(stat -c %a $file) # retrieve permission owner=$(stat -c %U $file) # retrieve owner group=$(stat -c %G $file) # retrieve group # just for debugging echo "$file@local: p = $permission, o = $owner, g = $group" # copy the permission ssh $user@$host "chmod $permission $file" < /dev/null # copy both owner and group ssh $user@$host "chown $owner:$group $file" < /dev/null done < list.txt I am assuming that the list of the files is saved in "list.txt". Moreover you should set the variables "user" and "host" accordingly to your setup. I would suggest to configure ssh to have "automatic login". Otherwise you should insert the password twice per loop. Here

Categories : Shell

ModelState.isValid = false on production machine, true on development machine
As I mentioned in the edit, I found the cause of and solution to my problem. It was caused by an error being thrown by the automatic binding of parameters, and the reason it was having issues with dates on my production machine was due to the locale settings (dd/mm/yy vs mm/dd/yy) on that machine. To fix, I set the computers region settings (in the control panel), and also added a line into my Web.config.

Categories : Jquery

Cannot find hadoop installation: $HADOOP_HOME must be set or hadoop must be in the path
Go to /etc/profile.d directory and create a hadoop.sh file in there with export HADOOP_HOME=/opt/hadoop/hadoop export HIVE_HOME=/opt/hadoop/hive export PATH=$PATH:$HADOOP_HOME/bin:$HIVE_HOME/bin After you save the file, make sure to chmod +x /etc/profile.d/hadoop.sh source /etc/profile.d/hadoop.sh This should take care of it.

Categories : Hadoop

how to Kill hadoop task started using hadoop jar command?
You can use the shell command kill. For example, use ps -ef | grep bigdat.twitter.crawler.CrawlTwitter to find the pid, and use kill -9 pid_of_the_process to kill it. You can write a script containing the following command to do the kill action: #!/bin/bash kill -9 $(ps -ef | grep bigdat.twitter.crawler.CrawlTwitter | sed "s/s+/ /g" | cut -f2)

Categories : Hadoop

Is it possible to remotely debug a 64 bit machine from 32 bit Host Machine(which has visual studio)
According to MSDN, the answer in the other article is incorrect, and you can debug x64 from an x86 host: http://msdn.microsoft.com/en-us/library/vstudio/ms184681%28v=vs.100%29.aspx If you are debugging remotely, Visual Studio can run under WOW64 or on a 32-bit computer. You can debug both IA64 and x64 applications, in addition to 32-bit applications that are running under x64 WOW mode or on 32-bit operating systems.

Categories : Dotnet

Use NLP / Machine Learning to teach a machine how to detect if a string is mathematical?
Parsing is a better tool than machine learning for this problem. What you have described is a relatively simple grammar for arithmetic, with some aliases for numbers, and a touch of syntax for those aliases. A tokenizer and some basic syntactic analysis, which you could code directly, will produce better more reliable results with significantly less computational effort than machine learning and optimization will. One reason why parsing is sufficient is that you don't need to worry about misspellings as often as you do, say, with people's names. If you want to get fancy about that, then use your Jaro-Winkler-based things for lexical analysis and then use syntatic analysis on what you think are your tokens. That is still much cheaper and less complex than machine learning. I don't know mu

Categories : PHP

MySQL statement failing on local machine but not deployment machine?
The failure is referring to the d.id in the nested subquery: ON dol.id=( SELECT MIN(dol2.id) FROM DOLineEJB dol2 WHERE d.id=dol2.DeliveryOrderEJB_lines ) You can fix the query: SELECT d.id, ..., dol.weight, d.status FROM DeliveryOrderEJB d join InbondEJB_DeliveryOrderEJB_link lnk on d.id=lnk.DeliveryOrderEJB_id left join DOLineEJB dol ON dol.id=( SELECT MIN(dol2.id) FROM DOLineEJB dol2 WHERE d.id=dol2.DeliveryOrderEJB_lines ) WHERE lnk.InbondEJB_itNo='...' ORDER BY d.id I believe the problem is because you have different versions of MySQL on the machines. MySQL changed the semantics of the , around version 5.0.

Categories : Mysql

PyDev: Running Code in local machine to remote machine
One solution would be: Install python on the remote machine Package your code into a python package using distutils (see http://wiki.python.org/moin/Distutils/Tutorial). Basically the process ends when you run the command python setup sdist in the root dir of your project, and get a tar.gz file in the dist/ subfolder. Copy your package to the remote server using scp, for example, if it is an amazon machine: scp -i myPemFile.pem local-python-package.tar.gz remote_user_name@remote_ip:remote_folder Run sudo pip install local-python-package.tar.gz on the remote server Now you can either SSH to the remote machine and run your code or use some remote enabler such as fabric to start commands on the remote server (works for any shell command, specifically python scripts) Alternatively, you c

Categories : Python

host webapp in my machine trough virtual machine
Yes to both questions, if your browser machine can hit the port the server machine is serving the app on. On WAN, you would need to make sure your router didn't block requests to the server's port, and the same for any modem/firewall that connects you to the internet at large. Also, if your public ip is not static, it may change.

Categories : Iis

How to calculate the offset of machine instructions using machine code itself?
In general, a machine instruction set uses three types of storage addresses: Absolute addresses which refer to the exact storage location of interest. Base-relative addresses which, when added to the contents of a "base register", refer to the storage location of interest. Instruction-relative addresses which, when added to the address of the instruction containing them, refer to the storage location of interest. Usually the 3rd type does not need to be "relocated", since the location being referred to will generally be in the same code segment as the referring instruction, so they move together. The second type may or may not require change upon binding/loading, depending on details of the compiled code binding scheme. The first type will almost always require changing, but it's u

Categories : Assembly



© Copyright 2017 w3hello.com Publishing Limited. All rights reserved.