When you upload a file to HDFS, you may encounter following exception.
Exception in thread “main” org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /testtxt. could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
If you are sure the capacity of data node is not exhausted and the communication between master node and data node is working well, then the issue may be caused by url resolving. Specifically, HDFS client cannot connect to datanode, although it can be connected with master node.
1. add following configuration on the org.apache.hadoop.conf.Configuration instance:
- Configuration conf = new Configuration();
- conf.set(“dfs.client.use.datanode.hostname”, “true”);
or add the following to hdfs-site.xml
2. confirm your client could ping datanode
E.g. datanode host name: quickstart.cloudera
If ping is failed, then add resolving mapping manually.
1) In Mac OS, use following command to edit hosts file.
2) add following mapping to the file.
Then, try your program again. The issue should be gone.