Why Automated Tests ?

Most of the companies are testing softwares for any platform manually with huge teams. Let’s check why they should change this processes as automated tests.

Reusable

Automated test codes are mostly reusable without any change of code line. Especially for regression tests which is boring for most of the testing teams. Previous tests are just away of one click 😉

No Time Restriction : 7 / 24 Available

Tests can be scheduled at any time , so it doesn’t require any extra time or effort for executing the tests.

Improves Testing Quality

Every step of the tests are defined in the scripts, so it is always runs with same code. Missing points are easily detected with the test results. So, it is easy to improve your test quality and also the overall quality of your software.

Bug Detection and Reporting

When the tests are executed manually, It is hard to reproduce cases and reporting the issue to developer teams. Hovewer, automated tests can detect bugs easily and reports to issue tracking systems automatically with step by step case, screenshots and error logs.

Reduces Costs

When the tests are automated ones, it just needs maintaning. No need to rework for every test cycles. So it reduces the testing costs.
In addition to this, your test team only adds new scenarios with new features.

Saves Time

Automated tests can run parallel for different platforms at the same time. So it saves the time , and has a positive effect on delivery time.

Increase Coverage

Test teams don’t spend their time for testing same scenarios / cases again and again. So that, they can implement new scenarios which can increase test coverages.

More Automation , More Momentum

If you want to make your software one step forward, you need to automate the every process.
Every automated step makes your software closer to the perfect one.

We are offering you to make the best with MOMENTUM which helps you to automate DevOps processes in terms of ;

• Building Versions
• Testing ( Unit, Integration, Functional )
• Code Quality & Coverage
• Deployment
• Issue Tracking

Doruk Coşkun
SW Test Developer

What is Test Automation? Manual Testing vs. Automated Tests

There are two main resources used to test software; humans and computers. While each resource has its own advantages and disadvantages; both aim to improve the quality of the software. In this article I would like to tell you about the basic differences between manual testing and test automation.

First of all, Manual Test is the simplest type of testing to find the bugs or anomalies in software. Essentially, it does not require any testing tool (despite there are many tools that can boost effectiveness). The main goal of the Manual Test is to make sure that the software is working according to the requirements. It can be called a “preliminary” test and, must be done before kickstarting more advanced testing, test automation.

Automated tests are mostly used to ensure a software is working, from an end-to-end perspective with the help of computers that can scale scenarios in different environments. Automated regression tests play a critical role, especially in transaction-based and high volume applications.

Not to create any confusion, automation and manual testing are not substitutes, but complement each other. While it is not always possible to cover each and every scenario with automation, manual testing is prone to human error. Automation needs be utilised with manual testing – if required. The feasibility of the test automation within the scope of the project and resource availability should be assessed carefully as an initial check.


Types of Testing

The following visuals show major testing types. Such tests can be done manually or by automation.

Black Box Testing: Such tests are not interested in code or design. Tests are about requirements and functionality. The black-box test is also called the Functionality test, the Closed Box test, or the Opaque test.

White Box Testing: Such tests depend on the knowledge of the internal logic of the application code. Elements, flow controls, conditions, etc. elements in the software code are tested.

Unit Testing: Small code blocks are tested and usually utilised by software developers responsible of the code itself.

Integration testing: After completing the unit test, the units or modules that enhance the integration test will be combined. The purpose of the integration test is to verify the functionality, performance and reliability of the integrated modules.

Acceptance testing: A test technique to determine if the software meets the required business specifications. The main purpose of this test is to verify whether the software meets the criteria for delivery to end-users.

Regression testing: testing the application end-to-end for any modifications or changes in functionality. It is difficult to include the entire system in the regression test manually, so automation tools are typically used for these types of tests.

Differences Between Manual Testing and Test Automation

In a project, depending on your unique conditions, you may choose to use only manual testing, or manual testing combined with test automation. You can find the summary as a guideline in the table below, which method to use in what situations.

Emine Güleç
Test Engineer

We welcome the latest Augmented Reality Kit from Apple: ARKit!

I will briefly talk about Apple’s ARKit, considered the latest phase of virtual reality.

Apple introduced the augmented reality program they worked on for some time at the WWDC. Craig Federighi, Apple’s VP of Software Engineering at conference, declared with the augmented reality developer kit (ARKit), developers can do many things from simple calculators to more complex visual shows on the iPhone and iPad. As an example, developers can “put coffee cups on the table” – these applications were already available – but now you can add the shadow of the cup and a book next to the cup. Voila!

We all know the Pokemon Go phenomenon. You can encounter a Pokemon while walking in your way and catch it by throwing a poke-ball. Tricky part is, now Pokemon may appear bigger or smaller, depending on your actual distance from its spawn point. This has never been applied with previous version of the technology available. You can actually walk around, or towards, or away from Pokemon fundamentally changing the entire encounter mechanic. A player now can approach from a different angle and distance to throw the poke-ball the Pokemon. Sounds cool!


IKEA is another example. Before you decide to buy a furniture, you can actually see its exact size and shape in your room. And questions of how much place it covers in your home/office or whether it fits will be a dilemma of the past!

Last but not the least example is real-time car catalogues (Audi and Mercedes rumoured to have plans already). Imagine looking to your car’s dashboard from your phone screen and you will instantly get information about that magic button. Or you are looking inside the hood to see which part and how to intervene when you want to add some oil – or some basics like how to open the hood if you’re a car rookie. In addition, when you want an extra feature for your car (sunroof, seat, steering wheel cover, navigation device, etc.) you can see how this fits. Are you also thinking of real-time ordering?

It is possible to say ARKit as the latest phase of virtual reality and Apple trusts on developers’ imagination. ARKit is free and generated great interest. Competition with ARKit through the new applications is now open and I think we will see them on the market soon.

One technical note: ARKit also support Unity, Unreal Engine and SceneKit, as well as Xcode application templates.

We look forward to working on your augmented reality idea. 🙂

Burak Tayfun
iOS Developer

Mobile-only banking…

Mobile technologies have evolved from pure communication handsets to tools that consumers use for banking, payments, shopping, entertainment and socializing. Their increasingly widespread use and their prevalence are changing the way that consumers access financial services, the way they make payments and make financial decisions in general. Turkey is in a privileged position to embrace digital financial changes, given its large young population, with high mobile penetration and a healthy financial sector, which has been at the forefront of financial innovation by quickly changing from channel orientation to prioritizing the customer.

Banks who can’t offer immediate mobile accessibility for their banking customers are falling behind the competition. Operating multiple branches is very costly when compared to the overheads associated with mobile-only challenger banks. As the number of mobile-only users continues to increase, banks will need to provide a ‘unique’ banking experience with great UI/UX and solid functionality.   

 
Turkey has a favorable environment for the adoption of new technologies and embracing digital changes. Turkey enjoys high mobile penetration and the number of Internet users is increasing over time although still below those in developed markets. The usage of smartphones is also picking up, use of online social media is also very popular, and e-commerce transactions continue to increase significantly. Therefore, the potential for e-banking and mobile banking is high, with customers receptive to enhanced customer experiences.
 
The Internet has changed the way we search for information and shop for products. Mobile technology has a far more wide-reaching impact, though, as in the developed world almost everyone has a mobile and in the developing world the usage of mobile is not only increasing quickly but it enables banks and businesses to reach rural and unbanked populations, accessing new customers. Increasing customer demands (they want things quickly, easily and right now) combined with ongoing competition from non-bank providers are driving the changes in the industry. For banks, mobile become most important channel for doing business, as it is essential to combine branch offerings with integrated digital services. Several Turkish banks have already changed their mind-sets, providing value-added services to ease daily banking actions with a customer-centric approach.

Causing the shift and switching of consumers in their droves to banks who provide a slick and intuitive mobile experience are ease of use, accessibility, mobile-only banks, switching to paperless, increased security.
 
As a conclusion, the banking and financial services’ sector are undergoing a significant transformation. One of the biggest drivers influencing this change is the deployment of new mobile technologies and innovations. Main key drivers are the rise of mobile-only banks, the availability of data driven money management tools, biometric authentications, fewer legal barriers to entry, the introduction of machine learning/AI and gamification into banking products, the ability for these services to create value for consumers. For banks already active in mobile app development this presents huge opportunities. For banks who have not yet developed a coherent mobile app development strategy, or for banks in the process of developing a strategy, there is now enormous pressure to develop mobile apps that can satisfy the very demanding banking consumers. Very often, the hardest piece of the puzzle for banks is working out how to develop the differentiating mobile apps that consumers demand.

Esra Peker
Sales Director

iOS Environment at Mobven

Once upon a time, iOS development tools were fairly simple and static. Objective C was a mature language which was founded on good old C. Cocoapods was only accepted dependency manager. Then Swift happened. No pointers, no semicolons, no .h files. It offered a safe haven for those who do not know C/C++ or Java. Swift 1.0 was a fun tool to use. Table/Collection views were no longer pain sources, projects became tidier and app prototyping time dramatically reduced. However, every change brings its mess along with it. Swift was no different. Unstable syntax (hail to those who witnessed the 2.0 migration), slow compiling times (like old Eclipse experience), buggy syntax highlighter (sorry not buggy, not working) and sleepless, caffeine-fueled engineers were the first I can write.

Swift is a product distilled from years of experience. Eventually, it will replace Objective-C in many fields but today is not that day. So, for not causing embarrassing production failures and petty crashes, we stick to the good old Objective – C and limit Swift in laboratory experiments. At Mobven, we created an app development process to give a developer an area of freedom.

The development process is fairly simple. The code is written along with unit tests. Then a senior reviews it with the original writer. If everything is OK, the code is merged into the main branch(called dev). CI system builds, tests and publishes with Fabric and cycle end.

To keep cycle in motion, tools must be fixed and reliable. Here is the 5 essential dev tools by Mobven iOS team.

Git: Source control is an indispensable tool for an engineer. It prevents the “new folder (42)” situation. Also, it keeps developers to work on the same codebase. Git is designed to manage enormous codebases and it does its job flawlessly.

Jenkins CI: Jenkins lifts the heavyweight of the cycle. Firstly, it pulls the code, run tests, creates an IPA and distributes it. Secondly, it checks every pushed branch for code quality and code coverage. Many developers hate to write unit tests like documentation but it is the failsafe of an entire application. Preventing to introduce new bugs into codebase is a huge plus and time-saving feature.

Cocoapods: As a dependency manager, Cocoapods works like Ruby Gems. It just works and saves countless hours. We use it both our applications and frameworks internally and publicly.

Capture: In app development, creating bug scenario is a daunting task. With Capture, our QA teams easily point the faulty UI implementations and crash situations.

Momentum: This in-house framework enables us to recreate bug scenarios without any effort. QA teams also use it for automated process test.

In conclusion, there is no one true way to create a mobile application. With consistency and discipline, our team creates an exemplary work. Momentum and Capture are the products of it.

 

Caner Y. Tatar
Software Engineer

Software Testing and Test Automation

Reviews are the greatest indication of the success of apps on the Google Play Store and the App Store. Interpretations and star counts are a measure of the total spent effort and the success/failure of the application. The reason for the majority of these negative comments is, that they are not correctly and well tested and also the testing process is not planned from the very beginning of the software. In this article, I would like to talk about the purpose of the testing and why it is important in the application development process.


Software Development Life Cycle (SDLC)
 

Software testing is an important part of the SDLC. And that’s why we should first summarize what the SDLC is. I think a test engineer should have information about the whole software process.

In a nutshell, SDLC is a cycle of requirements gathered from customers, analyzed, and then made up of design, development, testing, and publishing. In this cycle, the cost of software differs according to the place of error. The cost of the error caught in the last stages increases according to the the early stages. This makes the importance of the test higher.

A good test is to find serious mistakes early!

 

 

Testing and Software Testing

Testing is the period in which a system is tried or assessed manually or automatically to determine whether it meets specified requirements or to determine the difference between expected and observed results.

Software testing involves dynamically validating activities to meet the expected behavior of a software from an infinite number of work areas, limited number of tests and appropriately selected tests.


Why Do Software Testing?

* Ensure product quality before the customer is served,
* Reduce rework and development costs,
* To prevent the spreading of the development process in the early stages by detecting mistakes, thus saving time and cost,
* Made for general purposes like increasing customer satisfaction and preparing the ground for following orders.


Software Testing Techniques:

1. Black-Box Test: It is only a test of functionality and needs. They are also known as Closed Box, Functionality Test, or Opaque. Black-Box tests take into account the choice of input for the function of the operation to be performed and the expected or unexpected output of the operation to be performed.

In the black box test, the code is not examined and no special application or program is needed.

Tests that the user does not exist:

  • Ad-Hoc Test: Used to determine the duration and coverage of other tests.
  • Research Test: Used to learn the application before the test phase.
  • Regeneration Test: Used to understand how long the application will return to a fault.
  • Functionality Test: This is the testing of the functional requirements. If the expected results are obtained, the writing of the tests is continued.
  • Usability Test: A test that is performed in applications where the user interface is important.
  • Level Test: In the level test where the system end limits are tested, the system is forced to the end limits and this limit is determined.
  • Smoke Test: Smoke test determines that the application has successfully passed minor tests and is ready for major tests.
  • Stress Test: Monitoring the results of the same process performed many times at the same time. The durability and response of the application is determined.
  • Load Test: Used to determine under which conditions and where the system will crash.

Tests that the user exists:

  • User Acceptance Test: Examines whether the user has tested the system and met the expectations.
  • Alpha Test: A test where the developer calls the application development center to note the developer’s required information according to what they do in practice.
  • Beta Test: Tests that users who test the beta version of the deployed application examine the system and report feedback to the developer.

2. White Box Test
The white box test is based on the code of the application and tests the program’s execution and internal construction. In the white box test -also called glass, open box or clean box- the tester is examining the code to find the problematic part.

  • Unit Testing: Usually done by software developers. Small blocks of code are tested.
  • Static and Dynamic Analysis: Static analysis examines the code sequentially and searches for errors. Dynamic analysis examines the code’s operation and output.
  • Scope of Statement: Explanations in the code are tested. The accuracy of all statements is confirmed.
  • Security Testing: Testing of system security. Aims to identify system vulnerabilities such as unauthorized access, and to close these.
  • Change Test: This is a general test made after a problem in the system has been fixed.

 

Test Automation

 Software testing automation has taken place on stage as software industry has been placed in every sector. In an automation system, the total share of work determines the level of automation. Briefly, manual testing of software is done automatically through a script or a tool.

Although test automation is related to testing, it is actually a software development work. In the early stages of software development, it is necessary to start automation work. Successful test automation requires teamwork between test engineers and developers.


Advantages of Test Automation

  • Test automation procedures in an organization reduce the human-induced application of manual tests, resulting in better system testing.
  • Provides increased efficiency in the software testing process by identifying more errors earlier.
  • Automating repeated tests reduces testing costs.
  • The test engineer gains extra time to do more detailed testing through automation (Exploratory Testing and Usability Testing).
  • Infrastructural change plays an important role in the rapid completion of Regression Testing.
  • Facilitates reuse of tests.
  • Increase code coverage with tests.
  • Tests can run 7/24.
  • Provides increased reporting quality of tests.
  • Increases the quality of the product developed.

You can sip your cup of coffee while your tests are running:)

When to Run Test Automation?

It is necessary to start the automation business at an early stage; otherwise all of the test designs, test strategies, test result format, etc. should be changed, which increases cost and demotivation of the staff at the same time. In order to reduce these effects most, automation architecture should be done very well.

Automation should make things easier. Sometimes the most important faults are found with Ad Hoc tests, which means that the test taker puts itself in the customer’s place and tries various scenarios. So it detects important errors.

 

Emine Güleç
Test Engineer

Automated Mobile Testing

I think it is imperative to write an article on the problems we have experienced while developing mobile applications and the solutions we have found for these problems.

Process

Developing a mobile application is somewhat different than other software processes, giving you a product that can touch the end user, can see errors very clearly; I say ‘can touch’ because when you swipe on a list, there is a glitch in the application of the slipper.

Services and applications, running in the background of the device and the working duration can make your application run poorly, when it was working successfully in tests while you were developing the app.

The test processes of mobile applications and the time spent resolving these test outputs need not be underestimated at all, which is an inevitable consequence of a significant decline in motivation.

Because we are agile, we are pushing the test in parallel rather than leaving it at the end of the process, but  by the diversity of mobile devices and cores in every version are broken the previous system? The question frightened us.

While applying agile, there is a team presence in the heart of the process and obviously our team worked well and clean, but it can not guarantee that everything will be in order.

The situation should be marked as done after the test team has tested the modules written in the sprint according to the “definition of done” list and is sure that they work well on all devices.

Problem

Think of a team of 3 people working on a small project, one of them is a tester, checks the status of the application on all devices for two operating systems and does it over and over again.

This challenge led us to use test automation; first we looked at the solutions on the market, and at the end of this review, Appium, an open source solution, was more than we needed.

Appium was developed by Dan Cuellar as open source, who was a Test Manager in a company, looking for the exact need and couldn’t find the one.

We found the solution we searched, and we believed that this solution would solve all our problems.

Application

First of all, since a friend on the test team is familiar with Java language, we decided to go through the Java client solution.

We were using Junit and Eclipse Ide, scripts that were triggered on Ide by working on the phone connected to that computer; Finding the UI element, clicking on it, writing, swiping fastened our work, ass this was a really impressive beginning for us.

In the following days iOS testing requires Mac computers to test the team, and also hubs and cables to run multiple tests.

We switched these tests to NodeJ, which is said to work by spending little resources in every environment, to use a more flexible solution that we didn’t want to trigger the Ide.

Then we said we should work on all phones and shell scripts.

We wanted to schedule this work, to follow up on our reports, and we have completed the process by installing Jenkins.

Result

Now I look back and I do not understand how I thought that we completed the process :).

The setup was as following;

  • 7 test phones in Device Farm
  • Jenkins
  • Test-Scripts

Day after day the devices became insufficient, they were constantly plugged into the hub, they started to break, we could not manage our test scripts, we were stuck with the challenges created by a parallel run over Jenkins, the applications were somewhere in the test-case documents and the automation codes were somewhere else. We learned how difficult it would be to facilitate our work with automation, instead of getting easier.

Picture1 Picture2 Picture3 Picture4

One of the problems here is that we have forgotten that we are only a small part of the regularly established a test environment automation; and the other is, when you combine a lot of the parts that does not mean yo always create the Voltran .

Happy ending

We started from scratch, first of all we made our test processes more healthier with trainings, and made clear when we start the design tests, how we do the functional tests, where we activate the automation.

We wrote a panel that we could run on the Cloud and manage the whole process, we used Java Spring at the back, we preferred React.Js for the frontend.

We set up a special device-farm for this solution, where we made the platform we selected ourselves finalized with a lot of trials and errors.

We read published articles to create a diner, joined online trainings, and even found out that this work alone required a lot of work.

At the end, we have built a healthy structure with it’s own rooms, camera systems, air conditioning, internet, lighting and UPS systems.

Thanks to this solution and the experience we have gained through this solution, the customers, our test team and also me myself were happy.

Bonus

Then we realized, that we were not the only ones who lives these and  needs solution . So we have produced a solution, we have talked in panels, in fairs, and today we have become a product preferred by the biggest organizations in Turkey.

We took the opportunity to participate in domestic / foreign events and talked about our solution, we got prizes.

This product now has its own support, sales, etc. teams.

I’ve done a lot of other similar products like you’ve read a short story about stages of product development created from an important requirement; and they all have their own stories and I will share them in time.

Finally, I would like to add that I met Dan Cuellar, who I mentioned at the beginning of the article, and that I had the opportunity to talk about the solution and that he was very impressed.

Thank you..

 

Okan Ünverdi
CTO

Importance of effective bug reporting

Why trying to fix a ‘poorly reported bug’ reminds me of House M.D. (yes, the TV show)? 

For those who are not familiar with the popular TV show House MD, generally the plot goes like this:

  1. A patient showing bizarre symptoms is admitted and we quickly understand that if the cause of the symptoms can’t be discovered fast enough, the patient is going to die painfully.
  2. Theories from underdogs fly around and are checked one by one, generally under a lot of stress because the patient doesn’t have the time for all theories to be tested.
  3. House has a different approach, he looks where nobody looks, thinks what nobody thinks and prioritizes one theory above all others even it is not the seemingly logical one.
  4. House is right (of course) and the patient is cured. And we are explained the reason of House’s decision which is a small, sometimes environmental detail missed by everyone but him.

I have used this analogy because fixing bugs, especially in UAT, is a lot like a House episode. You have bugs (symptoms) that are reported that you have to fix, a customer breathing down your neck with a deadline (painful death), and multiple theories to fix the bug.

But who is our House? Who is the mystical figure who will discover the reason of the bug and apply the necessary treatment?

Well, truth to be told, we don’t have a House and since we don’t live in a TV drama series, we don’t believe there could be one such all-knowing figure. Instead a pretty rigorous discipline that we impose both on ourselves and customers: the discipline to report a bug in a well-documented way so that we have all the facts before starting the investigation.

What is “well reported”? For any developer to produce a reliable theory for the cause of a bug, he/she first has to reproduce it and to do that, he/she will need a number of elements which include internal and environmental information:

Build Version:

It is very critical for the developers to know on which build the bug was encountered. It may seem trivial, but you would be amazed to know the number of times the developer and the reporter are not working on the same build.

Preconditions:
To avoid longer steps, it is better to use “Preconditions” section effectively. You don’t need to go over and over again the steps that you’ve been through. Developers don’t like when the report looks unreadable. So, instead of listing out every single step to reach where the main issue lies, start your steps with “Preconditions” as mentioned below as an example:

– Login to your account.
– Navigate to “Update Profile” screen under “Settings” menu.
– Turn off Wi-Fi connection.

Steps to Reproduce (Actions Performed):

Make sure that your reproduction steps are clear, easy to read, in order, preferably numbered do not fail. Do not forget that a lack of clarity may lead to misunderstanding that can assigned back to the reporter as “Request more information.” which slows down the resolution process.

Example:

  1. Tap on “Upload Image” button, user is redirected to “Camera Roll” screen.
    2. Select an image, user is redirected to “Update Profile” screen.
    3. Make some changes on “First Name” and “Last Name” fields.
    4. Tap on “Next”.
    5. Observe that “You don’t seem to be connected to the internet.” warning message doesn’t appear.

Environment:

All too often, we see customers reporting bugs which are caused by connectivity issues; without checking that fact, your dev team can search for ages without being able to reproduce the bug and frustrate the customer in the process. Other environmental factors like OS version and phone model may turn out to be critical as well.

Example:

Tested with: iPhone 6S — iOS 9.0
Tested on: External Release — 1.0.0.90
Connection: Wifi / LTE/ 3G

Memory: %43

Attachments: Screenshots, Videos, Error Log Files

Attaching screenshots shows the context of the related issue quickly and helps to complement written details. Also, in case of a crash, you can pull the error logs and attach it to your report.

Not as easy as “the complete order button doesn’t work when I want to finish my order” right? Taking all the screenshots, annotating & comparing them and adding all the environment details with logs is no easy task but has to be done to avoid spending the time that you don’t have when your patient is dying.

We thought that if we couldn’t have House in the real world, we could have the next best thing: a tool that lays all the details and factors bare for the developer to see an developed Capture. Capture is a simple and easy to use bug reporting tool, which does all the legwork for the reporter and leaves him only the bug description. It takes screenshots, lets the reporter annotate them & add a description and when he is finished, gathers all the environmental details and logs and opens a Jira task automatically.

Capture was developed for internal purposes but we decided to bundle it as a product when we saw how useful it turned out to be for every organization developing mobile apps.

We can’t guarantee to heal your patients but we can guarantee that you will have all the information you need right before your eyes to do so.

For more information on Capture, visit
http://capture-mobile.com/

 

Cihan Ünür
Product Owner