Is there a way to deserialize the java object via python
and stored in hbase like ‘storing complete object (in byte[] format of above) in the Value portion of the KeyValue pair’ It is stored in hbase like (Example) column=container:container, timestamp=1480016194005, value=\xAC\xED\x00\x05sr\x00&com.test.container\x07\x89\x83\xFA\x7F\xD0F\xA5\x02\x00\x08I\x00\x07classIdJ\x00\x14dateTimeInLongFormatZ\x00\x04rootZ\x00\x09undefinedL\x00\x03keyt\x00\x12Ljava/lang/String;L\x00\x04modeq\x00~\x00\x01L\x00\x04nameq\x00~\x00\x01L\x00\x06userIdq\x00~\x00\x01xp\x00\x00\x00\x02\x00\x00\x01X\x967\xBA\xF0\x00\x00t\x00\x1Econtainer_393_5639181044834024t\x00\x06expandt\x00\x02ert\x00\x08testadmin when I try to retrieve the data, I used the following deserialization in java and converted back to object of readable format object = SerializationUtils.deserialize(bytes);
I would like to retrieve the data stored in java format via happybase using python and I achieved it and received the data as available in hbase like It is stored in hbase like (Example) column=container:container, timestamp=1480016194005, value=\xAC\xED\x00\x05sr\x00&com.test.container\x07\x89\x83\xFA\x7F\xD0F\xA5\x02\x00\x08I\x00\x07classIdJ\x00\x14dateTimeInLongFormatZ\x00\x04rootZ\x00\x09undefinedL\x00\x03keyt\x00\x12Ljava/lang/String;L\x00\x04modeq\x00~\x00\x01L\x00\x04nameq\x00~\x00\x01L\x00\x06userIdq\x00~\x00\x01xp\x00\x00\x00\x02\x00\x00\x01X\x967\xBA\xF0\x00\x00t\x00\x1Econtainer_393_5639181044834024t\x00\x06expandt\x00\x02ert\x00\x08testadmin Is there a way to deserialize the java object via python Thanks Much Hari
Passing java object to python
I am prototyping an interface to our application to allow other people to use python, our application is written in java. I would like to pass some of our data from the java app to the python code but I am unsure how to pass an object to python. I have done a simple java->python function call using simple parameters using Jython and found it very useful for what I am trying to do. Given the class below, how can I then use it in Python/Jython as an input to a function/class:
I had done a same type of application. What I done was I created an execuatable(exe) python code and passed my parameters value from java program . If this is ok for you then I will help. Please reply!!
3 Answers 3
One solution. You could use some sort of message system, queue, or broker of some sort to serialize/deserialize, or pass messages between python and java. Then create some sort workers/producer/consumers to put work on the queues to be processed in python, or java.
Also consider checking out for inspiration: https://www.py4j.org/
py4j is used heavily by/for pyspark and hadoop type stuff.
To answer your question more immediately.
import org.apache.commons.io.FileUtils; import org.json.simple.JSONObject; //import org.json.simple.JSONObject; public class TestObject < private double[] values; private int length; private int anotherVariable; private boolean someBool; private String someString; //getters, setters public String toJSON() < JSONObject obj=new JSONObject(); obj.put("values",new Double(this.values)); obj.put("length",new Integer(this.length)); obj.put("bool_val",new Boolean(this.SomeBool)); obj.put("string_key",this.someString); StringWriter out = new StringWriter(); obj.writeJSONString(out); return out.toString(); >public void writeObject() < Writer writer = new BufferedWriter( new OutputStreamWriter( new FileOutputStream("anObject.json"), "utf-8") ) ) writer.write(this.toJSON()); >public static void setObject() < values = 100.134; length = 12; anotherVariable = 15; someString = "spam"; >> class DoStuffWithObject(object): def __init__(self,obj): self.obj = obj self.changeObj() self.writeObj() def changeObj(self): self.obj['values'] = 100.134; self.obj['length'] = 12; self.obj['anotherVariable'] = 15; self.obj['someString'] = "spam"; def writeObj(self): ''' write back to file ''' with open('anObject.json', 'w') as f: json.dump(self.obj, f) def someOtherMethod(self, s): ''' do something else ''' print('hello <>'.format(s)) import json with open('anObject.json','r') as f: obj = json.loads(f.read()) # print out obj['values'] obj['someBool'] . for key in obj: print(key, objJava object in python) aThing = DoStuffWithObject(obj) aThing.someOtherMethod('there') And then in java read back the object. There are solutions that exist implementing this idea (JSON-RPC, XML-RPC, and variants). Depending, you may may also want to consider using something like ( http://docs.mongodb.org/ecosystem/drivers/java/ ) the benefit being that mongo does json.
A more comprehensive list of queues:
thats pretty comprehensive thanks. So this writes the JSON to a file? Would that work with large arrays (many thousands multidimensional).
As @Clashsoft points out the code itself has errors. I generally don’t program in java, and by generally I mean none. Apologies. But yes the idea is very much usable with thousands of arrays. However if it starts getting into a lot of objects being passed around, then you may want to consider how they are managed, hence the other resources that I have tried to link to.
Agree with the answer below. I think that the bottom line is that «Python and Java are separate interpreter-environments.» You therefore shouldn’t expect to transfer «an object» from one to the other. You shouldn’t expect to «call methods.» But it is reasonable to pass data from one to another, by serializing and de-serializing it through some intermediate data format (e.g. JSON) as you would do with any other program.
In some environments, such as Microsoft Windows, it’s possible that a technology like OLE (dot-Net) might be usable to allow environments to be linked-together «actively,» where the various systems implement and provide OLE-objects. But I don’t have any personal experience with whether, nor how, this might be done.
Therefore, the safest thing to do is to treat them as «records,» and to use serialization techniques on both sides. (Or, if you got very adventurous, run (say) Java in a child-thread.) An «adventurous» design could get out-of-hand very quickly, with little return on investment.
Decoding java objects in python
Is it possible to decode java objects in python? I know the structure of the java class that returns the data. The data is returned with the header of ‘Transfer-Encoding: chunked’ Or do I need to get the return data sent as xml / json? This is an example of the response I get; (Note some of the control characters are removed by pasting)
-- response -- 200 OK Server: Apache-Coyote/1.1 Transfer-Encoding: chunked Date: Tue, 22 Nov 2011 13:24:41 GMT ��srjava.util.ArrayListx����a�IsizexpGwXsr,com.blah.blah.data.ClientInfos����8d�LdbNametLjava/lang/String;Ldescriptionq~xptdbname1t Customername1q~tdbname2tCustomername2sq~t This example returns two clients in the format of client, databasename dbname1,customername1 dbname2,customername2
@zio: that’s not a «format». Did you serialize the objects (using DataOutputStream for example)? If so, then in theory you can de-serialize it, but it will be a lot trickier than if you used a sane data format in the first place.
1 Answer 1
— Edited as more details became available —
The content of this HTTP response is a serialized Java object.
Java serialization is a process by which an in-memory object gets packed into transport-friendly bytes for the purpose of being read by another JVM. In short, you need to read the serialization format. Assuming that they didn’t add a custom serializer, the actual protocol is documented here.
The rest is just assuring that you do a sensible Java to Python mapping of the data fields. If you want to support «round trip» data handling, you should cache the sometimes discarded «java information» in special «areas» of your data structure so you can serialize back to the same java constructs.
Either you are disassembling a class (which is easy to do as Java has a very rigid class structure and an easy to reverse byte code to source code mapping) or you are processing data.
If it is processing data, you’re not really decoding Java, and the ease of doing what you want depends heavily on the ability to gather knowledge about how the data is encoded. Since you mentioned a «Transfer-Encoding: Chunked» header, I am guessing you are reading a HTTP response (complete with headers).
Nearly every language has a library to handle HTTP responses due to the popularity of web services. Python already has a HTTP client as indicated here.