1. The vocabulary of conversational English is slightly different from the written one. People use a lot of slang.
2. The speed that people speak at is faster than someone learning English can keep up, and people often don't annunciate: they slur words together a lot.
3. Someone learning English as a second language usually needs to translate a sentence in order to understand (English to Chinese), and when responding, needs to translate a thought into the second language before saying it (Chinese to English).
4. When speaking, you want to say something perfectly.

The combination of the above makes it very hard to carry on a "normal" conversation.

Here is my advice:
1. If you want to improve, you must be willing to spend time in practicing it.
2. You need to first build a foundation, and this foundation is to be able to understand simple children's English without needing to translate.
3. The next step will be to expand your vocabulary to the adult level up to a point that you don't need to translate.

The final goal of the above is that you will think in English.

Do you think it is possible? I can tell you that it is! If you are living in an environment where everybody speaks English, it will take 6 months. If you don't have that environment, it will take longer, but it is still possible.

So my first question for you is: how much time are you willing to spend everyday in improving your English?