View source for BuBrain

<p size=5 align="center">BINGHAMTON UNIVERSITY</hp>
<h2 align="center">SPRING 2012</h2>
<h2 align="center">CS486</h2 >
<h2 align="center">BUBRAIN ANDROID PROJECT REPORT</h2 >

<h3 align="left">Akif Emin KOCAK</h3 >
<h3 align="left">Furkan OZDESLIK</h3 >
<h3 align="center">INDEX</h3 >

<h4 align="left">1.Introduction</h4>
<h4 align="left">2. Libraries and Technologies</h4>
<h4 align="left">3. Bubrain Android app</h4>
<h4 align="left">4. Improvements</h4>
<h4 align="left">5. Future plans</h4>
<h4 align="left">6. Conclusion</h4>
<h4 align="left">7. Reference</h4 >
<hr>
<h4 align="left">1.Introduction</h4>

<p>BuBrain android application project had started before we began it. Since we had taken the project, we have no idea about android or iPhone. Firstly, we have to examine the previous code that is written. We found many code repetitions, unnecessary long functions. We fixed long functions by decreasing the complexity as much as we can. But when we try to avoid repetitions, our program either didn’t  work or worked so poor. At this point we decided to change the system on project. We defined problems of the project on three matters to optimize it.</p>
<p>    Firstly we have to find a way to test it, because waiting for the dalvik (android avd) was stealing so much time. We should had found a way that we can test server requests. Firstly we found JUnit that is a java environment for online testing. It was really useful, but we needed to find out solutions of two more</p>
<p>    We need to find a way to clear html pages and take only the necessary objects that we needed. This was creating so many unnecessary processes. We find out HTMLCleaner library is created for that it has wide usage on mobile applications.</p>
<p>    Last thing is we needed to solve is to converting html tags with its information into android xml screen. Previous project was using substitute items on the view like whatever is necessary was just turning into visible at every page. It was a great performance failure due to generating all items first, and making them visible when the time comes. We searched different technologies and projects to see how do professional engineers do html to xml or html to iPhone. Finally we find out there is an industry standard in the market, which is called XPath. XPath is used to navigate through elements and attributes in an XML document. It was really useful, because we were able to clean previous code with some effort. Moreover, as It is industry standard, we thought it might be useful to use it for BuBrain iPhone application as well.  Although we couldn’t find so much documentation about it. It was really delightful that Firefox’s firebug add on is able to copy Xpath syntax directly from page.</p>

<p>    We will discuss the above at next section.</p>
<hr>
<h4 align="left">2. Libraries and Technologies</h4>


 <p> This project is heavily based on  parsing the BUBrain. That is why, we use two new  technologies which are HTMLCleaner and XPath. Also, for unit testing of our program, we use JUnit.</p>

<p align="center" >HTMLCleaner and XPath</p>
  
<p>  The common task of our smartphone app is parsing  a website. More clearly,   we parsing the BUBrain for information that our app use internally. However, Android API does not provide an easy way to do this. For that reason, many technologies have been developed to accomplish this task. Some people advocate parsing HTML pages like long strings, using regex’s or some other hard approaches. Some people support using a SAX parser( SAX threats HTML like XML).Actually that creates a problem because HTML is NOT XML. At the end, we decided to use free HTML parsing library (HTMLCleaner).However it comes with price that it does not fully support XPATH  features.</p>
    <p>What we did with HTMLCleaner is literally</p>
       <ul>
        <li>  Open BUBrain
        <li>  Programmatically extract information we need from it
        <li>  Put that information to our app.
       </ul>
   <p align="center">How to code HTMLCleaner</p>

  <p>Initialize the HTMLCleaner ,ans set its properties</p>
  <p> The first thing that we do is set up and HtmlCleaner object. We set a few properties for it, and then are ready to use it. We call its clean() method on the URL's input stream. This returns a TagNode for the root node in the document. A TagNode is a crucial part of the HtmlCleaner API: it represents a node in an XML document and you can use the API to work with its elements, attributes, and children nodes.</p>


<p><font color="#990000">
HtmlCleaner pageParser = new HtmlCleaner();
CleanerProperties props = pageParser.getProperties();
props.setAllowHtmlInsideAttributes(true);
props.setAllowMultiWordAttributes(true);
props.setRecognizeUnicodeChars(true);
props.setOmitComments(true);</font>
</p>
    <p>Open Connections</p>

<p><font color="#990000">URLConnection conn = url[0].openConnection();
TagNode node=pageParser.clean(new InputStreamReader(conn.getInputStream()));
</font></p>

   <p>XPath string for locating download links</p>
<p>   XPath says “Select out of all ‘’table”  elements with attribute “class” equals to ‘fileFormats’ with contains  element ‘tr’
Calling evaluateXPath() with this String basically says "return the set of all subnodes with table elements that have attribute class equal to file format which contain element tr"</p>

<p><font color="#990000">
String xPathExpression = "//table[@class='fileFormats']//tr";
</font></p>

<p>API retuns Object [] and nodes</p>
<p><font color="#990000">
Object[] downloadNodes = node.evaluateXPath(xPathExpression);</font></p>

    <p>At least, Get the data we want</p>

 <p><font color="#990000">
TagNode[] links = ((TagNode)linkNode).getElementsHavingAttribute("href", true);
ArrayList<string> stringLinks = new ArrayList<string>();</font></p>

<p>PS.  Like HTMLCleaner , we can use TagSoup ,HTML Parser.</p>
   
                                        <p align="center"> JUnit </p>

    <p>JUnit is generally used unit testing for Java. It is just like a toolkit which helps to develop good, robust test.</p>
       <p>keep the jUnit in different folder, because you don’t want to   mix two codes in same folder.</p>
 <p>then,  create java class</p>

<p><font color="#990000">package de.vogella.junit.first;
  public class MyClass {
 	public int multiply(int x, int y) {
 		return x / y; 	
   }
}</font></p>
<p>create a Junit test project</p>

<p>Create JUnit class</p>

<p><font color="#990000">package de.vogella.junit.first;
  import org.junit.Test;
  import static org.junit.Assert.assertEquals;
  public class MyClassTest {
  	@Test 	public void testMultiply() {
 		MyClass tester = new MyClass(); 		assertEquals("Result", 50, tester.multiply(10, 5));
 	}
 }</font></p>
<p>How to Run</p>
<p>The result of the tests will be displayed in the JUnit View.</p>
<hr>
<h4 align="left">3. BuBrain Android Application</h4>

	<p>As we mentioned above, we have to set up an environment to test our requests to server. After we set up the environment, we created a java class called “BuOnlineTest.java” to make our tests without using dalvik virtual machine. Basically, we fixed bubrain account to any user, and tested our functions by comparing the value that should be there and the value that comes back from server. Altough there are some little differences between Butest and bubrain server, It worked well for many functions. This is important because:</p>

<p>We were losing a lot of time by using android avd to test our variables.
We were losing a lot of time by checking log files that we created in order to check the variables that comes back from server.</p>

	<p>After we implemented Junit, Second step was the fixing unnecessary repetitions of code. In order to do that, We decided to implement htmlcleaner and Xpath. We created “BuParserPack.java” to fix the returns of xpath requests. Such as, we didnt want my money values to be string, but it has a format like “$1,234.00”. It is the class that convert it to “1234.00”. Moreover, We used some htmlcleaner in order to clean lines from unnecessary tags etc...</p>

	<p>“BuException.java” is the exception class that we created to throw our own exceptions.</p>

	<p>Our main class is “BuOnline.java”. Before we start to explain what this class is created for, We need to explain our “edu.binghamton.buonline.objects” package. This package  just consists of set, get and “toString” functions of the variables to that we need to pass on a required format. When a developer wants to create a new view on bubrain android, it is the first place to start. He/she needs to create a function at our main function that makes the request and take the data by using Xpath syntax. After function is created by the developer, setters and getters, and perhaps other convertions. We need this system in order to hold returning data as an object and  in a required format. If we go back to “BuOnline.java”, this  class is to make the post requests to the server and fix the responses with htmlcleaner and xpath. Every different kind of request should be handled here.</p>
	
	<p>“BuPostParams.java” is the class that we set the format of the post parameters.</p>
	
	<p>In summary, our program set the cookies and connection with login and password parameter. Then with the commands from the application, it starts to makes requests with using xpath, which targets directly to the variable needed. Htmlcleaner helps us to map our html objects. In a difference to previous version of the bubrain, new version is targeting variable needed without using effort to find it on html page. It makes it faster  compare to previous version, because previous version was making all requests on the android base. Now it is basically handled in Java project and the android project is fed by it. Moreover, It was harder to check if the request is correct or not. JUnit make it possible to not waste time on requests. We basicall handle all data transformation on network on the java. It makes us not dependent on android restrictions.</p>
<hr>
<h4 align="left">4. Improvements</h4>
	
 <p> The group before us create program so messy way. Each class can login, create cookie, set values, evaluate data, and  print the output. We think that this kind of programming crates problem because it violates some core design principles. </p>

  <p>Lets explain this with our UML diagram</p>

<p>However, in the other code,each class can not talk with each other.Program, starts with login,and by clicking buttons, classes  do their own jobs without any help of other classes.</p>
<hr>                  
<h4 align="left">5. Future plans</h4>
	
	<p>First thing to do is to finish all absent functions. There are some menu items, that doesn't work correctly due to lack of time. It takes so much time to makes all the requests. Unfortunately some commands that works with test server, doesn't work with real BuBrain server. We can say that after we decide how to do our project, it was a big problem that we had to change many functions that is not capable to work or work wrong in new server. It was even a bigger problem when we realize it late.</p>

	<p>Although android development is supported by more or less most of the java environment, it still has a lack of design tools. We followed the design of the previous project. We fixed the stuff that is obviously wrong at design. Restrictions didn't allow us to build better design. If a new design tools could be developed for android in the future, first thing to improve in program should be design.</p>

	<p>Finally, If BuBrain server administrators come up with a pack of server calls or  library that safely feed applications such as ours, this program may work so much faster. It is also possible to build a second server which works like “read-only” terminology, it is possible to take data quicklier with less code. On that conditions, it can also avoid many crashes on BuBrain servers as there will be less work to be handled especially in course registration period. This server may be provided from cloud with little expense. We can consider this as a second backup of the main server as well.</p>
<hr>
<h4 align="left">6. Conclusion</h4>
	
	<p>As we discussed before, this is not the right way to do a project like bubrain. Database access should be provided to the developer to maintain extreme optimization. Thus makes project not connection dependant only, also hardware dependent. Altough, many new phones as a greater memory and cpu, very first versions of android phones may have a lack of speed when working with this program. As computer world is speaking about a new terminology which is mobile programming that the memory usage and cpu usage must be carefully designed due to lack of sources on mobile platform.</p>

	<p>On the other hand, it is a really nice example of how to convert html into xml on a really efficient way. Htmlcleaner and xpath works perfectly together. We believe it might be great way of implementing portfolio or similar kind of websites in to android platform.</p>
<hr>
<h4 align="center">7. Screen Shots</h4>

<hr>
<h4 align="center">8. References</h4>
<br>
<ul>
<li>http://www.vogella.de/articles/JUnit/article.html</li>
<li>http://blog.andrewpearson.org/2010/07/android-html-parsing.html</li>
<li>http://stackoverflow.com/questions/8007333/java-android-xpath-html-parsing</li>
<li>http://thinkandroid.wordpress.com/2010/01/05/using-xpath-and-html-cleaner-to-parse-html-xml/</li>
<li>http://htmlcleaner.sourceforge.net/doc/index.html</li>
</ul>