Re: [ease-dev] [EASE] Sign and Verify Script

[Jonah Graham <jonah@xxxxxxxxxxxxxxxx>]

~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com


On 3 June 2016 at 06:54, Varun Raval <vraval48@xxxxxxxxx> wrote:
> Hi Jonah,
>
> Thanks for quick response. Here are my answers,
>
>> 1- I like how you choose to use Python's triple quotes to store signature.
>> Is there any example you have received this idea from. It may be useful for
>> future devs on the project to understand the rational for the choice there.
>
>
> I tried to keep signature block as similar as of javascript so that
> processing except for starting and ending lines are same.
> Reason for using multi-line comment is simple processing. In case of single
> line comment, we will have to remove comment character from each line to
> process, while in multi-line comment, that is not required.
Good idea. As I said I do like that idea. I was wondering if there are
other examples out there of someone doing the same/similar that you
derived the idea from, or have you come at this independently.


>> 3- In Extract_Sign_Cert_Content there are a few code review ideas I had,
>> would you like me to detail them?
>
>
> Yes, I would like review details. I would like to mention that methods need
> to be called in same order: sign, cert and content.
> content method can handle if it is called independently.

I'll try and find time, I can also review what you actually submit to
gerrit. Some highlevel items are part of the tests I recommended, more
detail below.

>
>> Here are a couple of tests to consider:
>>
>> 1- effects of newline characters. Especially if signature is created on
>> one platform and verified on another. I note that you are using readLine in
>> the code, so there is the possibility there of issues.
>
>
> While attaching signature, I have added \n characters at end of each line.
> See how signture is attached.
> As per the documentation of readLine() method of BufferedReader, readLine()
> will read the line and considers it terminated at \n or \r characters. Still
> to be sure, I will test it on other platforms also.
I am referring to newlines in the body, you do a readline, but then
you write a \n, what if readline has read a \r\n?

>
> I will test with slow streams.
>
>> I note that you are using ready to determine end of input, but that may
>> not do what you expect. What if the input is a StringReader for example?
>
>
> I did not understand this completely. What I can understand, I would not be
> able to perform readLine with StringReader. So we can convert StringReader
> to BufferedReader.

At the moment you are creating the filereader yourself, but presumably
in the real code you will deal with a generic reader or stream as
input? If your interface supports that you need to consider what
ready() method really does, it does not specify the end of input, only
that read will not block.

>
> On Fri, Jun 3, 2016 at 12:41 AM, Jonah Graham <jonah@xxxxxxxxxxxxxxxx>
> wrote:
>>
>> Hi Varun,
>>
>> Not sure I can answer any of your questions, but here are a few
>> thoughts/comments from reading a bit of your code. Please use them to guide
>> you as you see fit.
>>
>> 1- I like how you choose to use Python's triple quotes to store signature.
>> Is there any example you have received this idea from. It may be useful for
>> future devs on the project to understand the rational for the choice there.
>> 2- The signature passes pep8, flake8 and pylint default settings with no
>> warnings, that is very good.
>> 3- In Extract_Sign_Cert_Content there are a few code review ideas I had,
>> would you like me to detail them?
>>
>> Here are a couple of tests to consider:
>> 1- effects of newline characters. Especially if signature is created on
>> one platform and verified on another. I note that you are using readLine in
>> the code, so there is the possibility there of issues.
>> 2- test with input streams that are perhaps slow to respond, can input
>> stream provide all code up to and including signature, appear finished, but
>> then when run the rest of the (malicious) file after the signature runs
>> fine. I note that you are using ready to determine end of input, but that
>> may not do what you expect. What if the input is a StringReader for example?
>>
>> Jonah
>>
>>
>>
>> ~~~
>> Jonah Graham
>> Kichwa Coders Ltd.
>> www.kichwacoders.com
>>
>> On 2 June 2016 at 18:44, Varun Raval <vraval48@xxxxxxxxx> wrote:
>>>
>>> Hi,
>>>
>>> I have prepared more scripts with valid/invalid signature and this time
>>> in python and javascript. Scripts are available here.
>>>
>>> Format of signature is like this:
>>>
>>> At the end of script, a new line is added
>>> For javascript, Signature will start with /********BEGIN
>>> SIGNATURE********
>>>
>>> For python: """*****BEGIN SIGNSTURE********
>>>
>>> Then a signature in Base64 format is added
>>> A new line is added
>>> Certificate is encoded in Base64 and added
>>> A new line is added
>>> For javascript, Signature ends with ********END SIGNATURE********/
>>>
>>>  For python: ********BEGIN SIGNSTURE*****"""
>>>
>>> At time of verification, if there is any change in above format,
>>> IOException is thrown and signature format will be considered invalid and
>>> signature verification will fail.
>>>
>>> (a.) Original script is before new line added in first step above. So,
>>> for verification, this new line has to be excluded. So, malicious user might
>>> change this new line and add scripts instead. This needs to be prevented.
>>>
>>> Following are the causes due to which signature verification will fail
>>> i.e. signature is invalid
>>>
>>> Script modified after adding signature
>>> Script added in new line. see (a.)
>>> Starting line is in improper format
>>> A line is missing after signature
>>> Script is added after end of signature block
>>> signature or certificate blocks are modified
>>>
>>> (1) will cause verification error. i.e. signature is false
>>>
>>> (2), (3), (4), (5) will throw IOException.
>>>
>>> (6) may cause IOException or verification error depending upon how it is
>>> modified
>>>
>>> I am looking for more ways by which signature verification may fail. It
>>> would be great if someone can point out those I am missing.
>>>
>>> These scripts are signed by code that is written by me, available here.
>>> Core methods available here. I would like to generate more test data from
>>> some other signing utility but I am not able to find any. So if anyone knows
>>> about any other utility, I will be happy to use.
>>>
>>> getSign(), getCert(), getContent() methods in
>>> Extract_Sign_Cert_Content.java file are used to extract signature,
>>> certificate and main content over which signature is applied from script
>>> file.
>>>
>>>
>>> Sample signature with python script is attached here.
>>>
>>> Thanks,
>>>
>>>
>>> On Mon, May 30, 2016 at 12:56 AM Jonah Graham <jonah@xxxxxxxxxxxxxxxx>
>>> wrote:
>>>>
>>>> Hi Varun,
>>>>
>>>> Looks neat. I like the signature embedded in the script.
>>>>
>>>> As the resident Python advocate I just wanted to point out that the
>>>> comment format will need to be language dependent. Fortunately there are
>>>> already some utilities in EASE to deal with comments, so have a look at
>>>> org.eclipse.ease.ICodeParser and org.eclipse.ease.ICodeFactory, perhaps
>>>> extending them as needed if the functionality you need isn't there.
>>>>
>>>> Jonah
>>>>
>>>> ~~~
>>>> Jonah Graham
>>>> Kichwa Coders Ltd.
>>>> www.kichwacoders.com
>>>>
>>>> On 29 May 2016 at 19:28, Varun Raval <vraval48@xxxxxxxxx> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> My task of 23 May - 3 June is to create scripts with valid/invalid
>>>>> signatures for testing purposes and method to extract signature from script.
>>>>>
>>>>> I have prepared a sample script with signature which is attached here.
>>>>> The signature is created using a code that I have written using java
>>>>> security library and is here on Github. GetSigExistKeys.java contains
>>>>> methods for signing script. GenSigExistKeys.java shows how to use these
>>>>> methods.
>>>>>
>>>>> As I mentioned in a previous post, we can attach certificate with
>>>>> signature and script. Since certificate and signature are in binary format,
>>>>> they are converted to Base64 format and then attached to script.
>>>>>
>>>>> At the end of script, format is like this:
>>>>>
>>>>>  signature starts with /********BEGIN SIGNATURE********
>>>>> first line is the signature
>>>>> next set of lines is certificate in binary format
>>>>> signing part ends with ********END SIGNATURE********/
>>>>>
>>>>> Simple method to extract certificate from script is here on Github.
>>>>>
>>>>> If this format of signature is proper, I will create more scripts with
>>>>> valid/invalid signature.
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> On Sat, May 21, 2016 at 11:48 PM Varun Raval <vraval48@xxxxxxxxx>
>>>>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have used Decorators in Project Explorer to tell user the status of
>>>>>> that script: whether its signature is valid or invalid. I have attached a
>>>>>> screenshot of same below. Here, for example, .java file is having
>>>>>> broken(invalid) certificate decorator and .js file is having valid
>>>>>> decorator. Plugin is available on Github.
>>>>>>
>>>>>> We can control on what file to place which decorator by decorate()
>>>>>> method of class implementing ILightWeightLabelDecorator. For the first time
>>>>>> when file is executed, signature will be verified and result will be stored
>>>>>> using setPersistentProperty() method of IFile. Then each time, we can see
>>>>>> whether signature is valid or invalid by simply calling
>>>>>> getPersistentProperty().
>>>>>>
>>>>>> I was also able to solve the error of saving Preference Pages with
>>>>>> help of Jonah Graham. Updated plugin is on Github and screenshot is attached
>>>>>> here.
>>>>>>
>>>>>> I have posted major bugs on BugZilla with a parent bug [1] and other
>>>>>> child bugs are blocking this parent bug. Still several bugs are remaining to
>>>>>> be posted. I will be posting them soon.
>>>>>>
>>>>>> I was also experimenting on how to add Properties Page in Properties
>>>>>> menu of Context Menu. I have simply added two labels whose screenshot is
>>>>>> attached here. Again, plugin is on Github.
>>>>>>
>>>>>> I studied documentations on how to use CertPath classes [2], [3] for
>>>>>> getting certificate chain from Certificate file. From this certificate
>>>>>> chain, one can tell whether it is valid, signed by root i.e. public key
>>>>>> matches that of root, using CertPath class. CertificateFactory class is used
>>>>>> to instantiate a certificate from certificate file or from any input stream
>>>>>> [4].
>>>>>>
>>>>>> [1]: https://bugs.eclipse.org/bugs/show_bug.cgi?id=494099
>>>>>>
>>>>>> [2]:
>>>>>> https://docs.oracle.com/javase/7/docs/api/java/security/cert/CertPath.html
>>>>>> [3]:
>>>>>> https://docs.oracle.com/javase/7/docs/technotes/guides/security/certpath/CertPathProgGuide.html#CertPath
>>>>>> [4]:
>>>>>> https://docs.oracle.com/javase/7/docs/api/java/security/cert/CertificateFactory.html
>>>>>>
>>>>>>
>>>>>> On Sun, May 15, 2016 at 12:40 PM, Jonah Graham
>>>>>> <jonah@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>
>>>>>>> I have answered your question on SO, but repeated here for your
>>>>>>> convenience:
>>>>>>>
>>>>>>> Your problem is you haven't checked: Activate this plug-in when one
>>>>>>> of its classes is loaded in the MANIFEST.MF configuration. Checking that
>>>>>>> adds:
>>>>>>>
>>>>>>> Bundle-ActivationPolicy: lazy
>>>>>>>
>>>>>>> to your MANIFEST.MF and everything then works.
>>>>>>>
>>>>>>> When you have Bundle-ActivationPolicy: lazy it means you don't have
>>>>>>> to explicitly start your bundle, which is a good thing. In Eclipse you want
>>>>>>> to do as much lazily as possible and starting a bundle is a great example of
>>>>>>> that.
>>>>>>>
>>>>>>> Read more about tracking lifecyles or starting them.
>>>>>>>
>>>>>>>
>>>>>>> ~~~
>>>>>>> Jonah Graham
>>>>>>> Kichwa Coders Ltd.
>>>>>>> www.kichwacoders.com
>>>>>>>
>>>>>>> On 15 May 2016 at 06:05, Varun Raval <vraval48@xxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have made sample Preference Page which can be seen in Attachment
>>>>>>>> below. Problem with it is that once I enter values and close preferences and
>>>>>>>> reopen preferences, values are lost.
>>>>>>>>
>>>>>>>> I have followed tutorial on Vogella-Preferences. I have asked this
>>>>>>>> question on Stack-Overflow.
>>>>>>>>
>>>>>>>> Error is that Activator.start() is not called. I read on stack
>>>>>>>> overflow that Activator.start() is not called until something else in the
>>>>>>>> plugin is used on this link. Am I missing something here?
>>>>>>>>
>>>>>>>> Plugin project is on github.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> --
>>>>>>>> Varun Raval
>>>>>>>> _______________________________________________
>>>>>>>> ease-dev mailing list
>>>>>>>> ease-dev@xxxxxxxxxxx
>>>>>>>> To change your delivery options, retrieve your password, or
>>>>>>>> unsubscribe from this list, visit
>>>>>>>> https://dev.eclipse.org/mailman/listinfo/ease-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ease-dev mailing list
>>>>>>> ease-dev@xxxxxxxxxxx
>>>>>>> To change your delivery options, retrieve your password, or
>>>>>>> unsubscribe from this list, visit
>>>>>>> https://dev.eclipse.org/mailman/listinfo/ease-dev
>>>>>>
>>>>>>
>>>>> --
>>>>> Varun Raval
>>>>>
>>>>> _______________________________________________
>>>>> ease-dev mailing list
>>>>> ease-dev@xxxxxxxxxxx
>>>>> To change your delivery options, retrieve your password, or unsubscribe
>>>>> from this list, visit
>>>>> https://dev.eclipse.org/mailman/listinfo/ease-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> ease-dev mailing list
>>>> ease-dev@xxxxxxxxxxx
>>>> To change your delivery options, retrieve your password, or unsubscribe
>>>> from this list, visit
>>>> https://dev.eclipse.org/mailman/listinfo/ease-dev
>>>
>>> --
>>> Varun Raval
>>>
>>> _______________________________________________
>>> ease-dev mailing list
>>> ease-dev@xxxxxxxxxxx
>>> To change your delivery options, retrieve your password, or unsubscribe
>>> from this list, visit
>>> https://dev.eclipse.org/mailman/listinfo/ease-dev
>>
>>
>>
>> _______________________________________________
>> ease-dev mailing list
>> ease-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/ease-dev
>
>
>
> _______________________________________________
> ease-dev mailing list
> ease-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/ease-dev